Jul 22 03:31:27.080831 ip-10-0-139-52 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 22 03:31:27.080847 ip-10-0-139-52 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 22 03:31:27.080856 ip-10-0-139-52 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 22 03:31:27.081300 ip-10-0-139-52 systemd[1]: Failed to start Kubernetes Kubelet. Jul 22 03:31:37.314092 ip-10-0-139-52 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 22 03:31:37.314110 ip-10-0-139-52 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e3b3cb339531424cb5e6995619467ce8 -- Jul 22 03:33:33.093645 ip-10-0-139-52 systemd[1]: Starting Kubernetes Kubelet... Jul 22 03:33:33.480863 ip-10-0-139-52 kubenswrapper[2564]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 03:33:33.480863 ip-10-0-139-52 kubenswrapper[2564]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 03:33:33.480863 ip-10-0-139-52 kubenswrapper[2564]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 03:33:33.480863 ip-10-0-139-52 kubenswrapper[2564]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 03:33:33.480863 ip-10-0-139-52 kubenswrapper[2564]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 03:33:33.481540 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.480929 2564 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 03:33:33.486093 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486077 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:33:33.486093 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486093 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486098 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486102 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486105 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486108 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486110 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486113 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486116 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486119 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486121 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486124 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486126 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486129 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486131 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486134 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486136 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486139 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486142 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486147 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486150 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:33:33.486160 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486153 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486155 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486158 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486160 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486163 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486165 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486168 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486171 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486174 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486176 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486179 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486181 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486184 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486186 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486189 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486191 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486194 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486196 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486199 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486201 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:33:33.486643 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486204 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486206 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486209 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486211 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486213 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486216 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486220 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486223 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486225 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486228 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486232 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486236 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486239 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486241 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486254 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486258 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486261 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486265 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:33:33.487118 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486269 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486272 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486275 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486278 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486281 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486283 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486286 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486288 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486291 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486293 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486297 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486299 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486302 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486305 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486307 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486310 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486312 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486315 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486317 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486319 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:33:33.487566 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486322 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486325 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486327 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486330 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486346 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486349 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486351 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486354 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486357 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486359 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486362 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486365 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486367 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486370 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486372 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486375 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486377 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486380 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486382 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.486385 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:33:33.488196 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487090 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487097 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487102 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487105 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487108 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487111 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487114 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487117 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487119 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487122 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487125 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487127 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487131 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487134 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487136 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487139 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487141 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487144 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487146 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487148 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:33:33.488879 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487151 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487154 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487156 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487159 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487162 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487164 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487167 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487169 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487173 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487176 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487179 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487182 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487185 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487187 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487190 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487193 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487196 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487198 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487201 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487204 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:33:33.489435 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487207 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487210 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487214 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487216 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487219 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487222 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487224 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487227 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487230 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487233 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487236 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487239 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487244 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487247 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487249 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487252 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487254 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487259 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487261 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:33:33.489926 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487264 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487266 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487269 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487271 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487273 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487276 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487279 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487281 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487283 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487286 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487288 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487290 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487293 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487296 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487298 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487301 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487303 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487308 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487310 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487313 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:33:33.490409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487315 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487318 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487320 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487323 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487325 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487328 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487347 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487350 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487353 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487355 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487358 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487360 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487363 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487365 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487368 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487370 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487372 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487375 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487377 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.487379 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:33:33.490908 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489548 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489561 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489565 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489568 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489571 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489574 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489577 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489580 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489583 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489586 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489589 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489592 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489594 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489597 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489599 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489602 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489604 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489607 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489609 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:33:33.491403 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489611 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489614 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489616 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489619 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489621 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489624 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489629 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489640 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489643 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489645 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489648 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489650 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489653 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489656 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489659 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489662 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489665 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489668 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489670 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489673 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:33:33.491860 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489676 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489678 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489681 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489683 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489685 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489688 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489691 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489693 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489696 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489698 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489701 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489703 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489706 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489708 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489710 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489713 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489715 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489718 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489720 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489722 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:33:33.492346 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489725 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489728 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489731 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489733 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489736 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489738 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489741 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489744 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489747 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489749 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489752 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489754 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489757 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489759 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489761 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489763 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489766 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489769 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489771 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489773 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:33:33.492888 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489776 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489778 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489780 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489783 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489786 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489789 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489791 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489793 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489796 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489801 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489804 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489808 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489812 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489814 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489817 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489820 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489822 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489825 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:33:33.493391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489828 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.489830 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489913 2564 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489921 2564 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489928 2564 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489933 2564 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489937 2564 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489940 2564 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489944 2564 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489948 2564 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489951 2564 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489954 2564 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489957 2564 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489961 2564 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489964 2564 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489966 2564 flags.go:64] FLAG: --cgroup-root="" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489969 2564 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489972 2564 flags.go:64] FLAG: --client-ca-file="" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489975 2564 flags.go:64] FLAG: --cloud-provider="external" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489978 2564 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489983 2564 flags.go:64] FLAG: --cluster-domain="" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489986 2564 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489988 2564 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 03:33:33.493819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489991 2564 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489995 2564 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.489998 2564 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490001 2564 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490004 2564 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490007 2564 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490010 2564 flags.go:64] FLAG: --contention-profiling="false" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490013 2564 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490015 2564 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490019 2564 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490022 2564 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490026 2564 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490029 2564 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490033 2564 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490036 2564 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490038 2564 flags.go:64] FLAG: --enable-server="true" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490041 2564 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490046 2564 flags.go:64] FLAG: --event-burst="100" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490050 2564 flags.go:64] FLAG: --event-qps="50" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490052 2564 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490055 2564 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490058 2564 flags.go:64] FLAG: --eviction-hard="" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490062 2564 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490064 2564 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490067 2564 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 03:33:33.494418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490070 2564 flags.go:64] FLAG: --eviction-soft="" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490073 2564 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490075 2564 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490078 2564 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490081 2564 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490084 2564 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490087 2564 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490089 2564 flags.go:64] FLAG: --feature-gates="" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490093 2564 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490096 2564 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490099 2564 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490101 2564 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490107 2564 flags.go:64] FLAG: --healthz-port="10248" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490109 2564 flags.go:64] FLAG: --help="false" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490112 2564 flags.go:64] FLAG: --hostname-override="ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490115 2564 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490118 2564 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490121 2564 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490125 2564 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490128 2564 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490131 2564 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490133 2564 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490136 2564 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 03:33:33.495015 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490139 2564 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490142 2564 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490145 2564 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490148 2564 flags.go:64] FLAG: --kube-reserved="" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490150 2564 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490153 2564 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490156 2564 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490158 2564 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490161 2564 flags.go:64] FLAG: --lock-file="" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490164 2564 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490166 2564 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490169 2564 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490174 2564 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490177 2564 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490181 2564 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490183 2564 flags.go:64] FLAG: --logging-format="text" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490186 2564 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490189 2564 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490191 2564 flags.go:64] FLAG: --manifest-url="" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490194 2564 flags.go:64] FLAG: --manifest-url-header="" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490198 2564 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490202 2564 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490206 2564 flags.go:64] FLAG: --max-pods="110" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490209 2564 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490212 2564 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 03:33:33.495694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490215 2564 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490219 2564 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490222 2564 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490225 2564 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490228 2564 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490236 2564 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490238 2564 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490242 2564 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490245 2564 flags.go:64] FLAG: --pod-cidr="" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490247 2564 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490252 2564 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490255 2564 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490258 2564 flags.go:64] FLAG: --pods-per-core="0" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490261 2564 flags.go:64] FLAG: --port="10250" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490264 2564 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490266 2564 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0377508623174c8da" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490269 2564 flags.go:64] FLAG: --qos-reserved="" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490272 2564 flags.go:64] FLAG: --read-only-port="10255" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490275 2564 flags.go:64] FLAG: --register-node="true" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490278 2564 flags.go:64] FLAG: --register-with-taints="" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490282 2564 flags.go:64] FLAG: --registry-burst="10" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490284 2564 flags.go:64] FLAG: --registry-qps="5" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490287 2564 flags.go:64] FLAG: --reserved-cpus="" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490290 2564 flags.go:64] FLAG: --reserved-memory="" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490293 2564 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 03:33:33.496299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490296 2564 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490299 2564 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490302 2564 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490304 2564 flags.go:64] FLAG: --runonce="false" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490308 2564 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490311 2564 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490314 2564 flags.go:64] FLAG: --seccomp-default="false" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490317 2564 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490320 2564 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490323 2564 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490326 2564 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490329 2564 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490346 2564 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490350 2564 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490353 2564 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490355 2564 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490358 2564 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490361 2564 flags.go:64] FLAG: --system-cgroups="" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490364 2564 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490370 2564 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490373 2564 flags.go:64] FLAG: --tls-cert-file="" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490375 2564 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490379 2564 flags.go:64] FLAG: --tls-min-version="" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490382 2564 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490384 2564 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 03:33:33.496934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490387 2564 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490390 2564 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490393 2564 flags.go:64] FLAG: --v="2" Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490397 2564 flags.go:64] FLAG: --version="false" Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490401 2564 flags.go:64] FLAG: --vmodule="" Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490405 2564 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490408 2564 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490534 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490538 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490542 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490544 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490548 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490551 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490554 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490556 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490560 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490563 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490566 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490569 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490571 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490574 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:33:33.497589 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490576 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490579 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490582 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490584 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490587 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490590 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490592 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490595 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490597 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490600 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490603 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490605 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490608 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490611 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490613 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490616 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490618 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490623 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490626 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490628 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:33:33.498101 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490630 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490633 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490636 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490639 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490641 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490644 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490646 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490649 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490651 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490653 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490656 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490658 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490661 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490663 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490666 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490668 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490671 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490673 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490676 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490679 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:33:33.498615 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490681 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490684 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490686 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490689 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490691 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490693 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490696 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490698 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490700 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490704 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490710 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490712 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490715 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490719 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490723 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490726 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490729 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490731 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490734 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:33:33.499103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490736 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490739 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490741 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490744 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490746 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490749 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490751 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490753 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490756 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490758 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490760 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490763 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490765 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490768 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490770 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490772 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490775 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490779 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490782 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:33:33.499581 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490785 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490787 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490790 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490794 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490796 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490800 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.490803 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.490808 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.497178 2564 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.497194 2564 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.497215 2564 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.497221 2564 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497275 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497280 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497284 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:33:33.500027 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497287 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497290 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497293 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497296 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497299 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497302 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497305 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497307 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497310 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497313 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497315 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497319 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497323 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497326 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497329 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497348 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497351 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497353 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497356 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:33:33.500422 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497359 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497362 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497364 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497367 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497370 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497372 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497375 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497379 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497382 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497384 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497387 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497390 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497392 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497395 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497397 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497400 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497402 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497405 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497408 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497410 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:33:33.500906 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497413 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497415 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497418 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497420 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497422 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497425 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497427 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497430 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497432 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497435 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497438 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497440 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497443 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497445 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497448 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497450 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497453 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497455 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497458 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497461 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:33:33.501409 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497465 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497467 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497470 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497472 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497476 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497480 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497483 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497485 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497488 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497491 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497494 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497496 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497498 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497501 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497503 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497506 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497508 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497511 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497514 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497516 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:33:33.501891 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497519 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497521 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497524 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497526 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497529 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497531 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497534 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497537 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497539 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497542 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497545 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497547 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497552 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497556 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497559 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497561 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:33:33.502459 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497564 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.497569 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497693 2564 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497697 2564 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497700 2564 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497703 2564 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497705 2564 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497708 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497711 2564 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497714 2564 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497716 2564 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497719 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497722 2564 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497724 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 03:33:33.502837 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497727 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497729 2564 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497732 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497734 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497737 2564 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497739 2564 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497741 2564 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497744 2564 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497746 2564 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497749 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497751 2564 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497754 2564 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497756 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497759 2564 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497761 2564 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497764 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497766 2564 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497769 2564 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497771 2564 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497775 2564 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 03:33:33.503272 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497778 2564 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497782 2564 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497785 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497787 2564 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497790 2564 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497792 2564 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497795 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497798 2564 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497800 2564 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497803 2564 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497806 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497808 2564 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497810 2564 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497813 2564 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497815 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497818 2564 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497820 2564 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497822 2564 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497825 2564 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497827 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 03:33:33.503835 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497829 2564 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497832 2564 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497834 2564 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497836 2564 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497839 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497841 2564 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497844 2564 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497847 2564 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497849 2564 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497852 2564 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497854 2564 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497857 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497859 2564 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497862 2564 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497864 2564 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497866 2564 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497869 2564 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497871 2564 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497873 2564 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497876 2564 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 03:33:33.504533 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497878 2564 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497881 2564 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497883 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497885 2564 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497888 2564 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497890 2564 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497892 2564 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497895 2564 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497897 2564 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497899 2564 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497902 2564 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497904 2564 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497908 2564 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497911 2564 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497913 2564 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497916 2564 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497918 2564 feature_gate.go:328] unrecognized feature gate: Example Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497920 2564 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497923 2564 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 03:33:33.505314 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497926 2564 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497928 2564 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497930 2564 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497933 2564 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497936 2564 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497938 2564 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497940 2564 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:33.497943 2564 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.497947 2564 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.498050 2564 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.501686 2564 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.502941 2564 server.go:1019] "Starting client certificate rotation" Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.503055 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 03:33:33.505801 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.503099 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 03:33:33.506287 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.505301 2564 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 03:33:33.508036 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.508022 2564 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 03:33:33.522926 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.522901 2564 log.go:25] "Validated CRI v1 runtime API" Jul 22 03:33:33.524963 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.524951 2564 log.go:25] "Validated CRI v1 image API" Jul 22 03:33:33.525853 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.525837 2564 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 03:33:33.531005 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.530989 2564 fs.go:135] Filesystem UUIDs: map[61cd9338-def9-47bc-b985-397f7c81b5f8:/dev/nvme0n1p4 63e6669b-1c52-4541-ae83-fd9a6b2d0704:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 22 03:33:33.531056 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.531006 2564 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 03:33:33.539067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.538964 2564 manager.go:216] Machine: {Timestamp:2026-07-22 03:33:33.537541154 +0000 UTC m=+0.331029637 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3091622 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2dade636afa79039d640ddc580ca7a SystemUUID:ec2dade6-36af-a790-39d6-40ddc580ca7a BootID:e3b3cb33-9531-424c-b5e6-995619467ce8 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b6:c7:6f:eb:d3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b6:c7:6f:eb:d3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:1e:49:21:87:22:9e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 03:33:33.539067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.539057 2564 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 03:33:33.539184 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.539125 2564 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 03:33:33.541530 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.541513 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 03:33:33.542504 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.542483 2564 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 03:33:33.542652 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.542508 2564 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-52.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 03:33:33.543194 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.543185 2564 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 03:33:33.543223 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.543197 2564 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 03:33:33.543223 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.543204 2564 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 03:33:33.543817 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.543806 2564 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 03:33:33.545327 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.545316 2564 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 03:33:33.548181 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548171 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 22 03:33:33.548286 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548277 2564 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 03:33:33.548315 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548307 2564 kubelet.go:486] "Attempting to sync node with API server" Jul 22 03:33:33.548353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548317 2564 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 03:33:33.548353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548343 2564 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 03:33:33.548353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548352 2564 kubelet.go:398] "Adding apiserver pod source" Jul 22 03:33:33.548458 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548361 2564 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 03:33:33.548705 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548693 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 03:33:33.548752 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.548716 2564 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 03:33:33.551400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.551386 2564 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 03:33:33.553358 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.553321 2564 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 03:33:33.553405 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.553377 2564 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 03:33:33.554518 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554506 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554527 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554536 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554541 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554550 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554556 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554563 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554570 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554578 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 03:33:33.554584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554587 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 03:33:33.554812 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554606 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 03:33:33.554812 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.554611 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 03:33:33.555973 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.555962 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 03:33:33.555973 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.555973 2564 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 03:33:33.557686 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.557665 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 03:33:33.557750 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.557665 2564 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 03:33:33.559175 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.559162 2564 server.go:1268] "Started kubelet" Jul 22 03:33:33.559532 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.559353 2564 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 03:33:33.559532 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.559517 2564 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 03:33:33.559700 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.559563 2564 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 03:33:33.559976 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.559950 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jhhsx" Jul 22 03:33:33.560750 ip-10-0-139-52 systemd[1]: Started Kubernetes Kubelet. Jul 22 03:33:33.561095 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.561077 2564 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 03:33:33.562189 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.562169 2564 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-52.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 03:33:33.562307 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.562291 2564 server.go:310] "Adding debug handlers to kubelet server" Jul 22 03:33:33.566215 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.566197 2564 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 03:33:33.569020 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.569001 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jhhsx" Jul 22 03:33:33.569434 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.569415 2564 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 03:33:33.569434 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.569429 2564 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 03:33:33.569806 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.569675 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:33.570214 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570195 2564 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 03:33:33.570214 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570213 2564 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 03:33:33.570390 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570248 2564 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 03:33:33.570390 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570320 2564 reconstruct.go:97] "Volume reconstruction finished" Jul 22 03:33:33.570390 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570328 2564 reconciler.go:29] "Reconciler: start to sync state" Jul 22 03:33:33.570390 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570389 2564 factory.go:55] Registering systemd factory Jul 22 03:33:33.570614 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570401 2564 factory.go:223] Registration of the systemd container factory successfully Jul 22 03:33:33.570686 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.569649 2564 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-139-52.ec2.internal.18c47f33f9f3f972 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-139-52.ec2.internal,UID:ip-10-0-139-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-139-52.ec2.internal,},FirstTimestamp:2026-07-22 03:33:33.559134578 +0000 UTC m=+0.352623066,LastTimestamp:2026-07-22 03:33:33.559134578 +0000 UTC m=+0.352623066,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-139-52.ec2.internal,}" Jul 22 03:33:33.570762 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570751 2564 factory.go:153] Registering CRI-O factory Jul 22 03:33:33.570792 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570767 2564 factory.go:223] Registration of the crio container factory successfully Jul 22 03:33:33.570819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570814 2564 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 03:33:33.570854 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570844 2564 factory.go:103] Registering Raw factory Jul 22 03:33:33.570883 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.570861 2564 manager.go:1196] Started watching for new ooms in manager Jul 22 03:33:33.571545 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.571532 2564 manager.go:349] Starting recovery of all containers Jul 22 03:33:33.579353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.579319 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:33:33.582789 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.582647 2564 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-52.ec2.internal\" not found" node="ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.584720 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.584703 2564 manager.go:354] Recovery completed Jul 22 03:33:33.588594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.588580 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:33:33.591054 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.591038 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:33:33.591138 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.591080 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:33:33.591138 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.591095 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:33:33.591604 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.591592 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:33:33.591654 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.591592 2564 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 03:33:33.591654 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.591623 2564 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 03:33:33.591654 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.591641 2564 state_mem.go:36] "Initialized new in-memory state store" Jul 22 03:33:33.592295 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.592283 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:33:33.592390 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.592306 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:33:33.592390 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.592316 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:33:33.592860 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.592849 2564 policy_none.go:49] "None policy: Start" Jul 22 03:33:33.592891 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.592864 2564 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 03:33:33.592891 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.592873 2564 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 03:33:33.594096 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.594085 2564 policy_none.go:47] "Start" Jul 22 03:33:33.634877 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.634863 2564 manager.go:337] "Starting Device Plugin manager" Jul 22 03:33:33.634950 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.634890 2564 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 03:33:33.634950 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.634898 2564 server.go:87] "Starting device plugin registration server" Jul 22 03:33:33.635113 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.635094 2564 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 03:33:33.635218 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.635111 2564 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 03:33:33.635218 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.635178 2564 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 03:33:33.635329 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.635240 2564 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 03:33:33.635329 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.635249 2564 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 03:33:33.635737 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.635718 2564 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 03:33:33.635814 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.635759 2564 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:33.735577 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.735521 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:33:33.736677 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.736428 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:33:33.736677 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.736455 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:33:33.736677 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.736465 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:33:33.736677 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.736492 2564 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.737312 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.737294 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 03:33:33.738757 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.738741 2564 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 03:33:33.738832 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.738776 2564 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 03:33:33.738832 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.738792 2564 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 03:33:33.738832 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.738820 2564 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 03:33:33.740933 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.740910 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:33:33.744832 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.744817 2564 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.744902 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.744834 2564 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-52.ec2.internal\": node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:33.793857 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.793836 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:33.838895 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.838863 2564 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal"] Jul 22 03:33:33.838959 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.838946 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:33:33.839815 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.839798 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:33:33.839880 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.839829 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:33:33.839880 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.839839 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:33:33.841236 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.841224 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:33:33.841397 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.841381 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.841434 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.841425 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:33:33.842007 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.841990 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:33:33.842007 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.842005 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:33:33.842098 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.842020 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:33:33.842098 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.842034 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:33:33.842098 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.842023 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:33:33.842186 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.842101 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:33:33.843389 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.843374 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.843449 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.843401 2564 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 03:33:33.844196 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.844178 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientMemory" Jul 22 03:33:33.844265 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.844204 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 03:33:33.844265 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.844229 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeHasSufficientPID" Jul 22 03:33:33.857880 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.857856 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-52.ec2.internal\" not found" node="ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.862270 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.862253 2564 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-52.ec2.internal\" not found" node="ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.872501 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.872485 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68de950356ac2c91097b42c69a06b274-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal\" (UID: \"68de950356ac2c91097b42c69a06b274\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.872555 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.872509 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68de950356ac2c91097b42c69a06b274-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal\" (UID: \"68de950356ac2c91097b42c69a06b274\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.872555 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.872526 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd39ce5fb20c3f1559561494fcf3f4a1-config\") pod \"kube-apiserver-proxy-ip-10-0-139-52.ec2.internal\" (UID: \"bd39ce5fb20c3f1559561494fcf3f4a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.893915 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.893887 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:33.973267 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.973247 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68de950356ac2c91097b42c69a06b274-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal\" (UID: \"68de950356ac2c91097b42c69a06b274\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.973323 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.973247 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/68de950356ac2c91097b42c69a06b274-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal\" (UID: \"68de950356ac2c91097b42c69a06b274\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.973373 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.973300 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd39ce5fb20c3f1559561494fcf3f4a1-config\") pod \"kube-apiserver-proxy-ip-10-0-139-52.ec2.internal\" (UID: \"bd39ce5fb20c3f1559561494fcf3f4a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.973413 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.973368 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/bd39ce5fb20c3f1559561494fcf3f4a1-config\") pod \"kube-apiserver-proxy-ip-10-0-139-52.ec2.internal\" (UID: \"bd39ce5fb20c3f1559561494fcf3f4a1\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.973413 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.973383 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68de950356ac2c91097b42c69a06b274-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal\" (UID: \"68de950356ac2c91097b42c69a06b274\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.973511 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:33.973441 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/68de950356ac2c91097b42c69a06b274-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal\" (UID: \"68de950356ac2c91097b42c69a06b274\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:33.994082 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:33.994034 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.094817 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.094794 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.161981 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.161962 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:34.164914 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.164882 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" Jul 22 03:33:34.195080 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.195057 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.295644 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.295588 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.396147 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.396119 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.416951 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.416933 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:33:34.496466 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.496433 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.502599 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.502580 2564 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 03:33:34.502744 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.502713 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 03:33:34.502835 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.502753 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 03:33:34.502835 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.502758 2564 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 03:33:34.573946 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.573879 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 03:28:34 +0000 UTC" deadline="2028-03-22 06:42:31.558827225 +0000 UTC" Jul 22 03:33:34.573946 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.573915 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14619h8m56.984914505s" Jul 22 03:33:34.596814 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.596792 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.667939 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:34.667909 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd39ce5fb20c3f1559561494fcf3f4a1.slice/crio-27b96f792a9ac461a9c08695b4bbd00da7b703be15d032372cde6aa6199fdc68 WatchSource:0}: Error finding container 27b96f792a9ac461a9c08695b4bbd00da7b703be15d032372cde6aa6199fdc68: Status 404 returned error can't find the container with id 27b96f792a9ac461a9c08695b4bbd00da7b703be15d032372cde6aa6199fdc68 Jul 22 03:33:34.672714 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.672700 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:33:34.697636 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.697619 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.708814 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:34.708795 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68de950356ac2c91097b42c69a06b274.slice/crio-065b5167afabdb438e8b35d04284478c57315cdddd2716d1419864dfdc2219c9 WatchSource:0}: Error finding container 065b5167afabdb438e8b35d04284478c57315cdddd2716d1419864dfdc2219c9: Status 404 returned error can't find the container with id 065b5167afabdb438e8b35d04284478c57315cdddd2716d1419864dfdc2219c9 Jul 22 03:33:34.741577 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.741538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" event={"ID":"68de950356ac2c91097b42c69a06b274","Type":"ContainerStarted","Data":"065b5167afabdb438e8b35d04284478c57315cdddd2716d1419864dfdc2219c9"} Jul 22 03:33:34.742529 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.742508 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" event={"ID":"bd39ce5fb20c3f1559561494fcf3f4a1","Type":"ContainerStarted","Data":"27b96f792a9ac461a9c08695b4bbd00da7b703be15d032372cde6aa6199fdc68"} Jul 22 03:33:34.798327 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:34.798299 2564 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-139-52.ec2.internal\" not found" Jul 22 03:33:34.810913 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.810886 2564 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:33:34.869658 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.869598 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" Jul 22 03:33:34.883263 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.883243 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 03:33:34.884494 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.884479 2564 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" Jul 22 03:33:34.893888 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:34.893858 2564 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 03:33:35.475999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.475961 2564 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:33:35.548958 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.548927 2564 apiserver.go:52] "Watching apiserver" Jul 22 03:33:35.560479 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.560450 2564 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 03:33:35.560982 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.560950 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm","openshift-multus/multus-q5wp2","openshift-network-diagnostics/network-check-target-gkdws","openshift-network-operator/iptables-alerter-pbb7v","openshift-ovn-kubernetes/ovnkube-node-5pkln","kube-system/global-pull-secret-syncer-k9vqm","kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fd4r2","openshift-dns/node-resolver-fqjtc","openshift-image-registry/node-ca-4pgwk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal","openshift-multus/multus-additional-cni-plugins-7wddc","openshift-multus/network-metrics-daemon-m79rh","kube-system/konnectivity-agent-xxkvv"] Jul 22 03:33:35.562758 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.562737 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.563999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.563978 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:35.564117 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.564055 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:35.565390 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.565372 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:35.565500 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.565429 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:35.565985 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.565957 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 03:33:35.566515 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.566205 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 03:33:35.566515 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.566236 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-xbbs2\"" Jul 22 03:33:35.566515 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.566361 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 03:33:35.567662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.567641 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.569134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.569114 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.569877 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.569860 2564 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 03:33:35.570704 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.570683 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.570802 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.570717 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-rg47r\"" Jul 22 03:33:35.570802 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.570756 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:35.570802 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.570783 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 03:33:35.570959 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.570879 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 03:33:35.570959 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.570926 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:33:35.571778 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.571758 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 03:33:35.572168 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.572031 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.573461 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.573440 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-7wdj7\"" Jul 22 03:33:35.573551 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.573480 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 03:33:35.573712 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.573683 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.573781 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.573714 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 03:33:35.573781 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.573724 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 03:33:35.575402 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.575134 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 03:33:35.577064 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.576517 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 03:33:35.577064 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.575906 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 03:33:35.577064 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.576832 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 03:33:35.577064 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.576854 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pfckn\"" Jul 22 03:33:35.577290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.577171 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.578051 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.578032 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 03:33:35.578270 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.578251 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 03:33:35.579191 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.578587 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.579518 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.579464 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-4b8zd\"" Jul 22 03:33:35.580036 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580013 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26953cf6-9052-4b6c-adb7-7b3bc4e866af-host-slash\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.580174 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580158 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-ovn\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.580292 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580276 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/200a8fe2-eab4-4e9f-924f-03db74707208-kubelet-config\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.580415 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580379 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/200a8fe2-eab4-4e9f-924f-03db74707208-dbus\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.580538 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580511 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:35.580644 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-run-netns\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.580644 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580576 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovnkube-config\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.580762 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580724 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.580811 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580761 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-cni-bin\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.580811 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580787 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/db43e671-0bc5-403f-be13-99333644b038-serviceca\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.580909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580810 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jz29c\" (UniqueName: \"kubernetes.io/projected/db43e671-0bc5-403f-be13-99333644b038-kube-api-access-jz29c\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.580909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580837 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:35.580909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580864 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-etc-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.580909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580888 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovn-node-metrics-cert\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580912 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr6zn\" (UniqueName: \"kubernetes.io/projected/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-kube-api-access-wr6zn\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:35.581067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580937 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/26953cf6-9052-4b6c-adb7-7b3bc4e866af-kube-api-access-4494j\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.581067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580963 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.580990 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovnkube-script-lib\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581027 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26wgr\" (UniqueName: \"kubernetes.io/projected/10fe30fd-c5d3-46e9-abb0-b03312fd3174-kube-api-access-26wgr\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581067 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-slash\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581115 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-run-ovn-kubernetes\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581153 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-env-overrides\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581171 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.581353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581202 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db43e671-0bc5-403f-be13-99333644b038-host\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.581353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581235 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26953cf6-9052-4b6c-adb7-7b3bc4e866af-iptables-alerter-script\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.581353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581297 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-kubelet\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581368 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-systemd-units\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581396 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-systemd\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581440 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-var-lib-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581497 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-node-log\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581536 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581562 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-log-socket\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.581680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581609 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-cni-netd\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.582016 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581764 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 03:33:35.582016 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581878 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-llztn\"" Jul 22 03:33:35.582016 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.581917 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 03:33:35.582290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.582269 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.583818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.583242 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 03:33:35.583818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.583479 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wsn7d\"" Jul 22 03:33:35.583818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.583602 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 03:33:35.583818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.583670 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 03:33:35.584327 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.583918 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 03:33:35.584327 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.584124 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 03:33:35.584550 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.584526 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pnz6k\"" Jul 22 03:33:35.584674 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.584526 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 03:33:35.584674 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.584567 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:33:35.585581 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.585361 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-4487q\"" Jul 22 03:33:35.586049 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.585810 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 03:33:35.586049 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.585873 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 03:33:35.589544 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.589515 2564 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 03:33:35.613018 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.612999 2564 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pg2kb" Jul 22 03:33:35.619270 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.619244 2564 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pg2kb" Jul 22 03:33:35.671622 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.671598 2564 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 03:33:35.682467 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682441 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4sb8\" (UniqueName: \"kubernetes.io/projected/7fca8f59-2e5c-403a-adba-6d628d831737-kube-api-access-t4sb8\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.682566 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682471 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysctl-conf\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.682566 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682495 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x6mng\" (UniqueName: \"kubernetes.io/projected/8f61a890-2b9e-46a7-b95d-2fb102461e5f-kube-api-access-x6mng\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.682566 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682529 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr6zn\" (UniqueName: \"kubernetes.io/projected/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-kube-api-access-wr6zn\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:35.682726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682566 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682619 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fca8f59-2e5c-403a-adba-6d628d831737-tmp-dir\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.682726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682646 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-system-cni-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.682726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-system-cni-dir\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.682726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682690 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-kubelet\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.682726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682682 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682709 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-hostroot\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682769 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-66m9k\" (UniqueName: \"kubernetes.io/projected/a175ab40-db3f-4f61-8ccb-444d4a39af5f-kube-api-access-66m9k\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682813 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-kubelet\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682840 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-var-lib-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-node-log\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682907 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-netns\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682914 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-kubelet\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682954 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-node-log\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682957 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-var-lib-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.682999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.682992 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683034 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-cni-netd\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99146d59-9616-4c58-8989-c5b861c1b363-multus-daemon-config\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683086 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-cni-netd\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683145 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-sys\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683194 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-tuned\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683208 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64a8556c-948c-4704-b7c9-6c957eea69bc-tmp\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683222 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blt2x\" (UniqueName: \"kubernetes.io/projected/64a8556c-948c-4704-b7c9-6c957eea69bc-kube-api-access-blt2x\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683247 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26953cf6-9052-4b6c-adb7-7b3bc4e866af-host-slash\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683263 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-ovn\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683278 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/200a8fe2-eab4-4e9f-924f-03db74707208-dbus\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.683338 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683296 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xljwr\" (UniqueName: \"kubernetes.io/projected/99146d59-9616-4c58-8989-c5b861c1b363-kube-api-access-xljwr\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/26953cf6-9052-4b6c-adb7-7b3bc4e866af-host-slash\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-ovn\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683399 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cnibin\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683448 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-os-release\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683475 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/200a8fe2-eab4-4e9f-924f-03db74707208-dbus\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683478 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683516 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683543 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-run-netns\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683590 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-run-netns\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683621 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovnkube-config\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683651 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysconfig\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683671 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysctl-d\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-cni-bin\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683758 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-k8s-cni-cncf-io\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-multus-certs\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.684000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683825 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-cni-bin\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-etc-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683892 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovn-node-metrics-cert\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683912 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-etc-openvswitch\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683921 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-kubernetes\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-lib-modules\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.683979 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-var-lib-kubelet\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684033 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/26953cf6-9052-4b6c-adb7-7b3bc4e866af-kube-api-access-4494j\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.684064 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684179 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovnkube-script-lib\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.684229 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs podName:41c2a5d1-2687-40c4-b1dc-869e2f36ee58 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:36.184202087 +0000 UTC m=+2.977690575 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs") pod "network-metrics-daemon-m79rh" (UID: "41c2a5d1-2687-40c4-b1dc-869e2f36ee58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684270 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26wgr\" (UniqueName: \"kubernetes.io/projected/10fe30fd-c5d3-46e9-abb0-b03312fd3174-kube-api-access-26wgr\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684284 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovnkube-config\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684322 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684391 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-host\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684423 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.684830 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684453 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-slash\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-run-ovn-kubernetes\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684506 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-env-overrides\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684519 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-slash\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684533 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-os-release\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-cni-bin\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684580 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-host-run-ovn-kubernetes\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684588 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-sys-fs\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db43e671-0bc5-403f-be13-99333644b038-host\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26953cf6-9052-4b6c-adb7-7b3bc4e866af-iptables-alerter-script\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684780 2564 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684822 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db43e671-0bc5-403f-be13-99333644b038-host\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684805 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-systemd-units\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684842 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-systemd-units\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684800 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovnkube-script-lib\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684867 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-systemd\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684897 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-log-socket\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.685596 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684926 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/10fe30fd-c5d3-46e9-abb0-b03312fd3174-env-overrides\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684931 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-socket-dir-parent\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-socket-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685022 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-modprobe-d\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685047 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8590ac76-bbde-4b05-84c1-f3b01dfe0a13-agent-certs\") pod \"konnectivity-agent-xxkvv\" (UID: \"8590ac76-bbde-4b05-84c1-f3b01dfe0a13\") " pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fca8f59-2e5c-403a-adba-6d628d831737-hosts-file\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684993 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-run-systemd\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685096 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-cnibin\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.684974 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/10fe30fd-c5d3-46e9-abb0-b03312fd3174-log-socket\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685122 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99146d59-9616-4c58-8989-c5b861c1b363-cni-binary-copy\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685161 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-systemd\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685207 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/200a8fe2-eab4-4e9f-924f-03db74707208-kubelet-config\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685233 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8590ac76-bbde-4b05-84c1-f3b01dfe0a13-konnectivity-ca\") pod \"konnectivity-agent-xxkvv\" (UID: \"8590ac76-bbde-4b05-84c1-f3b01dfe0a13\") " pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685257 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-conf-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685263 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/200a8fe2-eab4-4e9f-924f-03db74707208-kubelet-config\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685283 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/26953cf6-9052-4b6c-adb7-7b3bc4e866af-iptables-alerter-script\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685306 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-registration-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.686244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685330 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-device-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685366 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-run\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685403 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-cni-multus\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-cni-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685446 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-etc-kubernetes\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/db43e671-0bc5-403f-be13-99333644b038-serviceca\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jz29c\" (UniqueName: \"kubernetes.io/projected/db43e671-0bc5-403f-be13-99333644b038-kube-api-access-jz29c\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.685495 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685511 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.685548 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret podName:200a8fe2-eab4-4e9f-924f-03db74707208 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:36.185529889 +0000 UTC m=+2.979018382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret") pod "global-pull-secret-syncer-k9vqm" (UID: "200a8fe2-eab4-4e9f-924f-03db74707208") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:35.686804 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.685872 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/db43e671-0bc5-403f-be13-99333644b038-serviceca\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.689009 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.688985 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/10fe30fd-c5d3-46e9-abb0-b03312fd3174-ovn-node-metrics-cert\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.698081 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.697969 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:33:35.698081 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.697993 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:33:35.698081 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.698007 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6tpv9 for pod openshift-network-diagnostics/network-check-target-gkdws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:35.698081 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:35.698058 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9 podName:2c29c9d9-c861-47a6-bc5d-4e9e316a5c09 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:36.198041173 +0000 UTC m=+2.991529660 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6tpv9" (UniqueName: "kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9") pod "network-check-target-gkdws" (UID: "2c29c9d9-c861-47a6-bc5d-4e9e316a5c09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:35.700798 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.700745 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr6zn\" (UniqueName: \"kubernetes.io/projected/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-kube-api-access-wr6zn\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:35.701855 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.701836 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4494j\" (UniqueName: \"kubernetes.io/projected/26953cf6-9052-4b6c-adb7-7b3bc4e866af-kube-api-access-4494j\") pod \"iptables-alerter-pbb7v\" (UID: \"26953cf6-9052-4b6c-adb7-7b3bc4e866af\") " pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.702282 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.702260 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26wgr\" (UniqueName: \"kubernetes.io/projected/10fe30fd-c5d3-46e9-abb0-b03312fd3174-kube-api-access-26wgr\") pod \"ovnkube-node-5pkln\" (UID: \"10fe30fd-c5d3-46e9-abb0-b03312fd3174\") " pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.702747 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.702725 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jz29c\" (UniqueName: \"kubernetes.io/projected/db43e671-0bc5-403f-be13-99333644b038-kube-api-access-jz29c\") pod \"node-ca-4pgwk\" (UID: \"db43e671-0bc5-403f-be13-99333644b038\") " pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.786148 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786083 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-cni-multus\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786148 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786127 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-cni-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786319 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786152 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-etc-kubernetes\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786319 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786205 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-etc-kubernetes\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786319 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786210 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-cni-multus\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786319 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786243 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-cni-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786319 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786271 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4sb8\" (UniqueName: \"kubernetes.io/projected/7fca8f59-2e5c-403a-adba-6d628d831737-kube-api-access-t4sb8\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.786319 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786301 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysctl-conf\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786327 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x6mng\" (UniqueName: \"kubernetes.io/projected/8f61a890-2b9e-46a7-b95d-2fb102461e5f-kube-api-access-x6mng\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786378 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fca8f59-2e5c-403a-adba-6d628d831737-tmp-dir\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786404 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-system-cni-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786429 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-system-cni-dir\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786455 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-kubelet\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786478 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-hostroot\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-66m9k\" (UniqueName: \"kubernetes.io/projected/a175ab40-db3f-4f61-8ccb-444d4a39af5f-kube-api-access-66m9k\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786531 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-netns\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786541 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-system-cni-dir\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786574 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-hostroot\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-netns\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786528 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-system-cni-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786616 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99146d59-9616-4c58-8989-c5b861c1b363-multus-daemon-config\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786644 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysctl-conf\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786652 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-kubelet\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.786662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786689 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-sys\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786707 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786714 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-tuned\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786722 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7fca8f59-2e5c-403a-adba-6d628d831737-tmp-dir\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786747 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64a8556c-948c-4704-b7c9-6c957eea69bc-tmp\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786774 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blt2x\" (UniqueName: \"kubernetes.io/projected/64a8556c-948c-4704-b7c9-6c957eea69bc-kube-api-access-blt2x\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786795 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xljwr\" (UniqueName: \"kubernetes.io/projected/99146d59-9616-4c58-8989-c5b861c1b363-kube-api-access-xljwr\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786800 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-sys\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786810 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cnibin\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-os-release\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786911 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786950 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysconfig\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786975 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysctl-d\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.786999 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-k8s-cni-cncf-io\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787030 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cnibin\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.787591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787055 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-multus-certs\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787086 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-kubernetes\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787114 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-lib-modules\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-var-lib-kubelet\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787167 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysctl-d\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787001 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-os-release\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787192 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/99146d59-9616-4c58-8989-c5b861c1b363-multus-daemon-config\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787231 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-k8s-cni-cncf-io\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787255 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787272 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-run-multus-certs\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787387 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-lib-modules\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787427 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-kubernetes\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787464 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-var-lib-kubelet\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787478 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-sysconfig\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787167 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787516 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-host\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787564 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-host\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787592 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.788588 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787636 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-os-release\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-cni-bin\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787686 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-sys-fs\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787692 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cni-binary-copy\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787712 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787754 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-socket-dir-parent\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787780 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-socket-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787804 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-modprobe-d\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787830 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8590ac76-bbde-4b05-84c1-f3b01dfe0a13-agent-certs\") pod \"konnectivity-agent-xxkvv\" (UID: \"8590ac76-bbde-4b05-84c1-f3b01dfe0a13\") " pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787856 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fca8f59-2e5c-403a-adba-6d628d831737-hosts-file\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787883 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-cnibin\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787906 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99146d59-9616-4c58-8989-c5b861c1b363-cni-binary-copy\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787932 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-systemd\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787937 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-host-var-lib-cni-bin\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787958 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8590ac76-bbde-4b05-84c1-f3b01dfe0a13-konnectivity-ca\") pod \"konnectivity-agent-xxkvv\" (UID: \"8590ac76-bbde-4b05-84c1-f3b01dfe0a13\") " pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787982 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8f61a890-2b9e-46a7-b95d-2fb102461e5f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.789510 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788052 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7fca8f59-2e5c-403a-adba-6d628d831737-hosts-file\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788071 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-sys-fs\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-conf-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788106 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-socket-dir-parent\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788128 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-registration-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788159 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-device-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-run\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-socket-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787907 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-os-release\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788280 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-multus-conf-dir\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788345 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-systemd\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.787857 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8f61a890-2b9e-46a7-b95d-2fb102461e5f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788371 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/99146d59-9616-4c58-8989-c5b861c1b363-cni-binary-copy\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788442 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-run\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788467 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/99146d59-9616-4c58-8989-c5b861c1b363-cnibin\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788492 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-device-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788507 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8590ac76-bbde-4b05-84c1-f3b01dfe0a13-konnectivity-ca\") pod \"konnectivity-agent-xxkvv\" (UID: \"8590ac76-bbde-4b05-84c1-f3b01dfe0a13\") " pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788560 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-modprobe-d\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.790097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.788654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a175ab40-db3f-4f61-8ccb-444d4a39af5f-registration-dir\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.790784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.789723 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/64a8556c-948c-4704-b7c9-6c957eea69bc-tmp\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.790784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.790099 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/64a8556c-948c-4704-b7c9-6c957eea69bc-etc-tuned\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.790921 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.790893 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8590ac76-bbde-4b05-84c1-f3b01dfe0a13-agent-certs\") pod \"konnectivity-agent-xxkvv\" (UID: \"8590ac76-bbde-4b05-84c1-f3b01dfe0a13\") " pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.798637 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.798612 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-66m9k\" (UniqueName: \"kubernetes.io/projected/a175ab40-db3f-4f61-8ccb-444d4a39af5f-kube-api-access-66m9k\") pod \"aws-ebs-csi-driver-node-lk6nm\" (UID: \"a175ab40-db3f-4f61-8ccb-444d4a39af5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.798763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.798734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blt2x\" (UniqueName: \"kubernetes.io/projected/64a8556c-948c-4704-b7c9-6c957eea69bc-kube-api-access-blt2x\") pod \"tuned-fd4r2\" (UID: \"64a8556c-948c-4704-b7c9-6c957eea69bc\") " pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.798834 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.798734 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x6mng\" (UniqueName: \"kubernetes.io/projected/8f61a890-2b9e-46a7-b95d-2fb102461e5f-kube-api-access-x6mng\") pod \"multus-additional-cni-plugins-7wddc\" (UID: \"8f61a890-2b9e-46a7-b95d-2fb102461e5f\") " pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:35.800845 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.800821 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4sb8\" (UniqueName: \"kubernetes.io/projected/7fca8f59-2e5c-403a-adba-6d628d831737-kube-api-access-t4sb8\") pod \"node-resolver-fqjtc\" (UID: \"7fca8f59-2e5c-403a-adba-6d628d831737\") " pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.800937 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.800870 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xljwr\" (UniqueName: \"kubernetes.io/projected/99146d59-9616-4c58-8989-c5b861c1b363-kube-api-access-xljwr\") pod \"multus-q5wp2\" (UID: \"99146d59-9616-4c58-8989-c5b861c1b363\") " pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.878814 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.878783 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4pgwk" Jul 22 03:33:35.888684 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.888655 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-pbb7v" Jul 22 03:33:35.898695 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.898669 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:33:35.904515 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.904495 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-fqjtc" Jul 22 03:33:35.912240 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.912221 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:35.920290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.920269 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q5wp2" Jul 22 03:33:35.926185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.926167 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" Jul 22 03:33:35.932432 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.932411 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" Jul 22 03:33:35.937056 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:35.937032 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7wddc" Jul 22 03:33:36.077251 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.077180 2564 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:33:36.153369 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.153343 2564 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 03:33:36.190865 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.190836 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:36.190999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.190913 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:36.191059 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.191004 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:36.191059 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.191028 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:36.191155 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.191077 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs podName:41c2a5d1-2687-40c4-b1dc-869e2f36ee58 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:37.191056134 +0000 UTC m=+3.984544626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs") pod "network-metrics-daemon-m79rh" (UID: "41c2a5d1-2687-40c4-b1dc-869e2f36ee58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:36.191155 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.191097 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret podName:200a8fe2-eab4-4e9f-924f-03db74707208 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:37.191087757 +0000 UTC m=+3.984576238 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret") pod "global-pull-secret-syncer-k9vqm" (UID: "200a8fe2-eab4-4e9f-924f-03db74707208") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:36.292076 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.292036 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:36.292216 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.292147 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:33:36.292216 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.292162 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:33:36.292216 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.292171 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6tpv9 for pod openshift-network-diagnostics/network-check-target-gkdws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:36.292326 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.292218 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9 podName:2c29c9d9-c861-47a6-bc5d-4e9e316a5c09 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:37.292203682 +0000 UTC m=+4.085692153 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tpv9" (UniqueName: "kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9") pod "network-check-target-gkdws" (UID: "2c29c9d9-c861-47a6-bc5d-4e9e316a5c09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:36.621208 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.621136 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 03:28:36 +0000 UTC" deadline="2028-01-16 18:55:00.818027991 +0000 UTC" Jul 22 03:33:36.621208 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.621200 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13047h21m24.196832804s" Jul 22 03:33:36.684233 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.684204 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99146d59_9616_4c58_8989_c5b861c1b363.slice/crio-4202c48e0b12a1c99a39d4f49c3a1e67edaf7ecf1ed9758a06b3a7eda35cc177 WatchSource:0}: Error finding container 4202c48e0b12a1c99a39d4f49c3a1e67edaf7ecf1ed9758a06b3a7eda35cc177: Status 404 returned error can't find the container with id 4202c48e0b12a1c99a39d4f49c3a1e67edaf7ecf1ed9758a06b3a7eda35cc177 Jul 22 03:33:36.690918 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.690868 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fca8f59_2e5c_403a_adba_6d628d831737.slice/crio-fffb2f9c84bd5441c2ad2c81bdfa320c72006c5b3da9badf9f7ed8cc5db41c81 WatchSource:0}: Error finding container fffb2f9c84bd5441c2ad2c81bdfa320c72006c5b3da9badf9f7ed8cc5db41c81: Status 404 returned error can't find the container with id fffb2f9c84bd5441c2ad2c81bdfa320c72006c5b3da9badf9f7ed8cc5db41c81 Jul 22 03:33:36.692103 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.692082 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda175ab40_db3f_4f61_8ccb_444d4a39af5f.slice/crio-922e936a5cd83f0bd33e9a647c46074923358dbf46be78d94a3ec6a7b59dac2f WatchSource:0}: Error finding container 922e936a5cd83f0bd33e9a647c46074923358dbf46be78d94a3ec6a7b59dac2f: Status 404 returned error can't find the container with id 922e936a5cd83f0bd33e9a647c46074923358dbf46be78d94a3ec6a7b59dac2f Jul 22 03:33:36.701197 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.701171 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10fe30fd_c5d3_46e9_abb0_b03312fd3174.slice/crio-063b54a9ae20321beb47c21a513fc0bd34c3d52e91642f006fd25337ce15e43e WatchSource:0}: Error finding container 063b54a9ae20321beb47c21a513fc0bd34c3d52e91642f006fd25337ce15e43e: Status 404 returned error can't find the container with id 063b54a9ae20321beb47c21a513fc0bd34c3d52e91642f006fd25337ce15e43e Jul 22 03:33:36.702544 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.702513 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f61a890_2b9e_46a7_b95d_2fb102461e5f.slice/crio-481757b7b974f83c06e1542778cb29ec13b0f7a06db3c32958dc78fcf461daf8 WatchSource:0}: Error finding container 481757b7b974f83c06e1542778cb29ec13b0f7a06db3c32958dc78fcf461daf8: Status 404 returned error can't find the container with id 481757b7b974f83c06e1542778cb29ec13b0f7a06db3c32958dc78fcf461daf8 Jul 22 03:33:36.703311 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.703286 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a8556c_948c_4704_b7c9_6c957eea69bc.slice/crio-a90a12c35266d62f2996a85faaf8655b9218443bc551690523ebadf9f887cae0 WatchSource:0}: Error finding container a90a12c35266d62f2996a85faaf8655b9218443bc551690523ebadf9f887cae0: Status 404 returned error can't find the container with id a90a12c35266d62f2996a85faaf8655b9218443bc551690523ebadf9f887cae0 Jul 22 03:33:36.704986 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.704962 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb43e671_0bc5_403f_be13_99333644b038.slice/crio-413f77abe22cbe48996686e7ea3b630f5ceefc64a8ca9cf7f6a222c45dcc35f2 WatchSource:0}: Error finding container 413f77abe22cbe48996686e7ea3b630f5ceefc64a8ca9cf7f6a222c45dcc35f2: Status 404 returned error can't find the container with id 413f77abe22cbe48996686e7ea3b630f5ceefc64a8ca9cf7f6a222c45dcc35f2 Jul 22 03:33:36.706952 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.706908 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26953cf6_9052_4b6c_adb7_7b3bc4e866af.slice/crio-47f996a151dd42664ef48de265c65bb1976916df4085abd6a622aed65fb2b1a2 WatchSource:0}: Error finding container 47f996a151dd42664ef48de265c65bb1976916df4085abd6a622aed65fb2b1a2: Status 404 returned error can't find the container with id 47f996a151dd42664ef48de265c65bb1976916df4085abd6a622aed65fb2b1a2 Jul 22 03:33:36.707994 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:33:36.707936 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8590ac76_bbde_4b05_84c1_f3b01dfe0a13.slice/crio-d7b30f1449b1857c4b7ba9b658239413403f55e7abe9701f4457537f74921b9d WatchSource:0}: Error finding container d7b30f1449b1857c4b7ba9b658239413403f55e7abe9701f4457537f74921b9d: Status 404 returned error can't find the container with id d7b30f1449b1857c4b7ba9b658239413403f55e7abe9701f4457537f74921b9d Jul 22 03:33:36.739148 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.738986 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:36.739251 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:36.739179 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:36.746393 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.746367 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerStarted","Data":"481757b7b974f83c06e1542778cb29ec13b0f7a06db3c32958dc78fcf461daf8"} Jul 22 03:33:36.747538 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.747513 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4pgwk" event={"ID":"db43e671-0bc5-403f-be13-99333644b038","Type":"ContainerStarted","Data":"413f77abe22cbe48996686e7ea3b630f5ceefc64a8ca9cf7f6a222c45dcc35f2"} Jul 22 03:33:36.748922 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.748898 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"063b54a9ae20321beb47c21a513fc0bd34c3d52e91642f006fd25337ce15e43e"} Jul 22 03:33:36.749996 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.749965 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" event={"ID":"a175ab40-db3f-4f61-8ccb-444d4a39af5f","Type":"ContainerStarted","Data":"922e936a5cd83f0bd33e9a647c46074923358dbf46be78d94a3ec6a7b59dac2f"} Jul 22 03:33:36.751678 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.751650 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fqjtc" event={"ID":"7fca8f59-2e5c-403a-adba-6d628d831737","Type":"ContainerStarted","Data":"fffb2f9c84bd5441c2ad2c81bdfa320c72006c5b3da9badf9f7ed8cc5db41c81"} Jul 22 03:33:36.753730 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.753707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q5wp2" event={"ID":"99146d59-9616-4c58-8989-c5b861c1b363","Type":"ContainerStarted","Data":"4202c48e0b12a1c99a39d4f49c3a1e67edaf7ecf1ed9758a06b3a7eda35cc177"} Jul 22 03:33:36.755306 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.755282 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xxkvv" event={"ID":"8590ac76-bbde-4b05-84c1-f3b01dfe0a13","Type":"ContainerStarted","Data":"d7b30f1449b1857c4b7ba9b658239413403f55e7abe9701f4457537f74921b9d"} Jul 22 03:33:36.756723 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.756661 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pbb7v" event={"ID":"26953cf6-9052-4b6c-adb7-7b3bc4e866af","Type":"ContainerStarted","Data":"47f996a151dd42664ef48de265c65bb1976916df4085abd6a622aed65fb2b1a2"} Jul 22 03:33:36.757994 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:36.757970 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" event={"ID":"64a8556c-948c-4704-b7c9-6c957eea69bc","Type":"ContainerStarted","Data":"a90a12c35266d62f2996a85faaf8655b9218443bc551690523ebadf9f887cae0"} Jul 22 03:33:37.198669 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.198584 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:37.198832 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.198668 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:37.198832 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.198812 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:37.198947 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.198872 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs podName:41c2a5d1-2687-40c4-b1dc-869e2f36ee58 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:39.198853266 +0000 UTC m=+5.992341755 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs") pod "network-metrics-daemon-m79rh" (UID: "41c2a5d1-2687-40c4-b1dc-869e2f36ee58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:37.199319 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.199295 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:37.199454 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.199353 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret podName:200a8fe2-eab4-4e9f-924f-03db74707208 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:39.199337442 +0000 UTC m=+5.992825928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret") pod "global-pull-secret-syncer-k9vqm" (UID: "200a8fe2-eab4-4e9f-924f-03db74707208") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:37.300192 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.299931 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:37.300192 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.300065 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:33:37.300192 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.300090 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:33:37.300192 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.300104 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6tpv9 for pod openshift-network-diagnostics/network-check-target-gkdws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:37.300192 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.300164 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9 podName:2c29c9d9-c861-47a6-bc5d-4e9e316a5c09 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:39.300145382 +0000 UTC m=+6.093633874 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tpv9" (UniqueName: "kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9") pod "network-check-target-gkdws" (UID: "2c29c9d9-c861-47a6-bc5d-4e9e316a5c09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:37.622320 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.622194 2564 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 03:28:36 +0000 UTC" deadline="2027-12-25 02:41:16.362966838 +0000 UTC" Jul 22 03:33:37.622320 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.622232 2564 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12503h7m38.740739126s" Jul 22 03:33:37.742443 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.742357 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:37.742614 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.742515 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:37.742997 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.742924 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:37.743097 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:37.743029 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:37.770860 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.770825 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" event={"ID":"bd39ce5fb20c3f1559561494fcf3f4a1","Type":"ContainerStarted","Data":"8ae575aa5564ffc59d9328f23cc8722a54208c4d9c65c813db3a0099a24994c6"} Jul 22 03:33:37.777089 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.777059 2564 generic.go:356] "Generic (PLEG): container finished" podID="68de950356ac2c91097b42c69a06b274" containerID="860d75bfbcabdae7c03a5e2a3b1491f5cd08148eed2c098439d4c8f48596c9f3" exitCode=0 Jul 22 03:33:37.777187 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.777102 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" event={"ID":"68de950356ac2c91097b42c69a06b274","Type":"ContainerDied","Data":"860d75bfbcabdae7c03a5e2a3b1491f5cd08148eed2c098439d4c8f48596c9f3"} Jul 22 03:33:37.794039 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:37.793988 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-52.ec2.internal" podStartSLOduration=2.793968869 podStartE2EDuration="2.793968869s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:37.793939496 +0000 UTC m=+4.587427993" watchObservedRunningTime="2026-07-22 03:33:37.793968869 +0000 UTC m=+4.587457357" Jul 22 03:33:38.739383 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:38.739348 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:38.739907 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:38.739502 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:38.794103 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:38.794065 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" event={"ID":"68de950356ac2c91097b42c69a06b274","Type":"ContainerStarted","Data":"0d15c560b3ccae41a6f3e24e79d1b5fe31743b63e9c249b84b23bde2448f917b"} Jul 22 03:33:38.816485 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:38.816295 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-52.ec2.internal" podStartSLOduration=3.816273791 podStartE2EDuration="3.816273791s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:38.815029449 +0000 UTC m=+5.608517943" watchObservedRunningTime="2026-07-22 03:33:38.816273791 +0000 UTC m=+5.609762290" Jul 22 03:33:39.219026 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:39.218929 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:39.219026 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:39.219003 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:39.219256 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.219139 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:39.219256 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.219202 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret podName:200a8fe2-eab4-4e9f-924f-03db74707208 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:43.219181938 +0000 UTC m=+10.012670425 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret") pod "global-pull-secret-syncer-k9vqm" (UID: "200a8fe2-eab4-4e9f-924f-03db74707208") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:39.219638 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.219618 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:39.219739 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.219677 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs podName:41c2a5d1-2687-40c4-b1dc-869e2f36ee58 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:43.219659486 +0000 UTC m=+10.013147962 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs") pod "network-metrics-daemon-m79rh" (UID: "41c2a5d1-2687-40c4-b1dc-869e2f36ee58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:39.319990 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:39.319887 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:39.320167 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.320073 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:33:39.320167 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.320098 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:33:39.320167 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.320112 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6tpv9 for pod openshift-network-diagnostics/network-check-target-gkdws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:39.320341 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.320183 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9 podName:2c29c9d9-c861-47a6-bc5d-4e9e316a5c09 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:43.320164996 +0000 UTC m=+10.113653475 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tpv9" (UniqueName: "kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9") pod "network-check-target-gkdws" (UID: "2c29c9d9-c861-47a6-bc5d-4e9e316a5c09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:39.740661 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:39.739881 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:39.740661 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.740031 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:39.740661 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:39.740507 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:39.740661 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:39.740624 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:40.739972 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:40.739938 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:40.740172 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:40.740091 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:41.739525 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:41.739493 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:41.739972 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:41.739644 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:41.739972 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:41.739495 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:41.740118 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:41.740011 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:42.739735 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:42.739706 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:42.740151 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:42.739813 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:43.251006 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:43.250963 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:43.251175 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:43.251039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:43.251175 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.251163 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:43.251276 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.251217 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret podName:200a8fe2-eab4-4e9f-924f-03db74707208 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:51.251199703 +0000 UTC m=+18.044688174 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret") pod "global-pull-secret-syncer-k9vqm" (UID: "200a8fe2-eab4-4e9f-924f-03db74707208") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:43.251578 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.251560 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:43.251712 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.251610 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs podName:41c2a5d1-2687-40c4-b1dc-869e2f36ee58 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:51.251595284 +0000 UTC m=+18.045083758 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs") pod "network-metrics-daemon-m79rh" (UID: "41c2a5d1-2687-40c4-b1dc-869e2f36ee58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:43.351611 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:43.351553 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:43.351795 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.351718 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:33:43.351795 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.351732 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:33:43.351795 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.351741 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6tpv9 for pod openshift-network-diagnostics/network-check-target-gkdws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:43.351795 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.351779 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9 podName:2c29c9d9-c861-47a6-bc5d-4e9e316a5c09 nodeName:}" failed. No retries permitted until 2026-07-22 03:33:51.351767653 +0000 UTC m=+18.145256123 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tpv9" (UniqueName: "kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9") pod "network-check-target-gkdws" (UID: "2c29c9d9-c861-47a6-bc5d-4e9e316a5c09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:43.741236 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:43.740735 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:43.741236 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.740859 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:43.741236 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:43.740910 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:43.741236 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:43.741019 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:44.739191 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:44.739147 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:44.739389 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:44.739275 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:45.739811 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:45.739778 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:45.740203 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:45.739930 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:45.740203 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:45.739983 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:45.740203 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:45.740083 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:46.739157 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:46.739122 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:46.739324 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:46.739246 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:47.738996 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:47.738971 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:47.739354 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:47.738981 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:47.739354 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:47.739073 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:47.739354 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:47.739184 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:48.739927 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:48.739892 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:48.740319 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:48.740009 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:49.739619 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:49.739583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:49.739769 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:49.739591 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:49.739769 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:49.739702 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:49.739845 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:49.739795 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:50.739346 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:50.739316 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:50.739753 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:50.739451 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:51.311952 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:51.311919 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:51.312205 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:51.311974 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:51.312205 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.312077 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:51.312205 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.312127 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs podName:41c2a5d1-2687-40c4-b1dc-869e2f36ee58 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:07.312113049 +0000 UTC m=+34.105601519 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs") pod "network-metrics-daemon-m79rh" (UID: "41c2a5d1-2687-40c4-b1dc-869e2f36ee58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:33:51.312205 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.312077 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:51.312407 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.312221 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret podName:200a8fe2-eab4-4e9f-924f-03db74707208 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:07.312202978 +0000 UTC m=+34.105691466 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret") pod "global-pull-secret-syncer-k9vqm" (UID: "200a8fe2-eab4-4e9f-924f-03db74707208") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:33:51.413011 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:51.412976 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:51.413170 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.413156 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:33:51.413245 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.413175 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:33:51.413245 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.413188 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6tpv9 for pod openshift-network-diagnostics/network-check-target-gkdws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:51.413245 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.413239 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9 podName:2c29c9d9-c861-47a6-bc5d-4e9e316a5c09 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:07.413223192 +0000 UTC m=+34.206711665 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tpv9" (UniqueName: "kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9") pod "network-check-target-gkdws" (UID: "2c29c9d9-c861-47a6-bc5d-4e9e316a5c09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:33:51.739861 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:51.739790 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:51.740300 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.739915 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:51.740300 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:51.739972 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:51.740300 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:51.740060 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:52.739300 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:52.739272 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:52.739492 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:52.739394 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:53.739698 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:53.739667 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:53.740160 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:53.739754 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:53.740160 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:53.739779 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:53.740160 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:53.739865 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:54.739964 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:54.739937 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:54.740426 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:54.740039 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:55.739356 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:55.739308 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:55.739613 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:55.739479 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:55.739613 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:55.739526 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:55.739713 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:55.739640 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:56.740019 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.739727 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:56.740883 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:56.740114 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:56.835916 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.835884 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" event={"ID":"64a8556c-948c-4704-b7c9-6c957eea69bc","Type":"ContainerStarted","Data":"180d97f66052592fa52bb3f544c418c6db06623061ebcacfaea7c44b8b3b3b81"} Jul 22 03:33:56.837771 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.837731 2564 generic.go:356] "Generic (PLEG): container finished" podID="8f61a890-2b9e-46a7-b95d-2fb102461e5f" containerID="fc7431e106818bd15fa190eaebff9b9498c063a081d3a039b922235478964569" exitCode=0 Jul 22 03:33:56.837901 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.837820 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerDied","Data":"fc7431e106818bd15fa190eaebff9b9498c063a081d3a039b922235478964569"} Jul 22 03:33:56.839749 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.839729 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4pgwk" event={"ID":"db43e671-0bc5-403f-be13-99333644b038","Type":"ContainerStarted","Data":"ab4b8c61c43c2c5e9a9b260ba187265d239e8f4d1f7806a9a309028ac7efe03f"} Jul 22 03:33:56.844030 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.844005 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"5abd136da1ce44c5cc6a2a4fa2f70080ae14c29496d31e9349d0f493efacc1a6"} Jul 22 03:33:56.844152 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.844082 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"6769bc40cecadcfcfaa2ef3c1e02eec64c705b69878db906f4c89fdd2148c772"} Jul 22 03:33:56.844152 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.844131 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"1a5912efd5d172f89dae066412623d881806613ea756b331aa2be5f36337b66d"} Jul 22 03:33:56.844237 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.844177 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"c1d13265d085b7ced85643feeab92ccfed34d056ec0b1258a129e80c33e5dfeb"} Jul 22 03:33:56.844237 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.844224 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"248957819f15c8bbdf213cbe1ed1938b078a6c2a45df23f70322b6447331b152"} Jul 22 03:33:56.844359 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.844272 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"ce7b908e2b1cc9a2d1f7a5e1f3afbea2ed3873a1dc1987a128957d5e0e3a95c6"} Jul 22 03:33:56.845590 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.845567 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" event={"ID":"a175ab40-db3f-4f61-8ccb-444d4a39af5f","Type":"ContainerStarted","Data":"a45186aa1884cd3e42526ee96ac145c678638d8fbfb469ba3b58ecc0c5e40fed"} Jul 22 03:33:56.847040 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.847007 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-fqjtc" event={"ID":"7fca8f59-2e5c-403a-adba-6d628d831737","Type":"ContainerStarted","Data":"bae5f6706c5775f9220e12f278e6c1eb636cbc64ea82da9edcc84b624d83d4ac"} Jul 22 03:33:56.848656 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.848625 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q5wp2" event={"ID":"99146d59-9616-4c58-8989-c5b861c1b363","Type":"ContainerStarted","Data":"3c0c33b63ee16a5258b720db1be6ed4f9a951ac7c9dca9c4e22a840cc55cba68"} Jul 22 03:33:56.850296 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.850275 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xxkvv" event={"ID":"8590ac76-bbde-4b05-84c1-f3b01dfe0a13","Type":"ContainerStarted","Data":"87f19cf489364bc57180e4faa6154417b7a20ed6b0f2ec0e8cbc56422de69d48"} Jul 22 03:33:56.853984 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.853948 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fd4r2" podStartSLOduration=3.805650412 podStartE2EDuration="22.853937093s" totalImagesPullingTime="19.048286681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:56.853446856 +0000 UTC m=+23.646935352" watchObservedRunningTime="2026-07-22 03:33:56.853937093 +0000 UTC m=+23.647425584" Jul 22 03:33:56.868169 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.868119 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-fqjtc" podStartSLOduration=3.809999404 podStartE2EDuration="22.868103942s" totalImagesPullingTime="19.058104538s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:56.867713724 +0000 UTC m=+23.661202209" watchObservedRunningTime="2026-07-22 03:33:56.868103942 +0000 UTC m=+23.661592435" Jul 22 03:33:56.883172 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.883126 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xxkvv" podStartSLOduration=11.848005259 podStartE2EDuration="22.883110127s" totalImagesPullingTime="11.035104868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:56.882747091 +0000 UTC m=+23.676235583" watchObservedRunningTime="2026-07-22 03:33:56.883110127 +0000 UTC m=+23.676598622" Jul 22 03:33:56.900482 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.900450 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q5wp2" podStartSLOduration=3.778441027 podStartE2EDuration="22.900439555s" totalImagesPullingTime="19.121998528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:56.899894012 +0000 UTC m=+23.693382504" watchObservedRunningTime="2026-07-22 03:33:56.900439555 +0000 UTC m=+23.693928046" Jul 22 03:33:56.917606 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:56.917563 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4pgwk" podStartSLOduration=3.875304008 podStartE2EDuration="22.917550371s" totalImagesPullingTime="19.042246363s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:56.914573977 +0000 UTC m=+23.708062470" watchObservedRunningTime="2026-07-22 03:33:56.917550371 +0000 UTC m=+23.711038862" Jul 22 03:33:57.326368 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.326320 2564 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 03:33:57.653703 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.653558 2564 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T03:33:57.326360013Z","UUID":"aa81107d-3b93-4561-a0df-3fec8af431db","Handler":null,"Name":"","Endpoint":""} Jul 22 03:33:57.655211 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.655192 2564 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 03:33:57.655330 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.655232 2564 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 03:33:57.739300 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.739271 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:57.739469 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.739275 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:57.739469 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:57.739422 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:57.739618 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:57.739540 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:57.854194 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.854163 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-pbb7v" event={"ID":"26953cf6-9052-4b6c-adb7-7b3bc4e866af","Type":"ContainerStarted","Data":"dcfec1868d10057f05f4bb6288f08a5bfc20d61862175030fdbecc25bba0c070"} Jul 22 03:33:57.856170 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.856120 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" event={"ID":"a175ab40-db3f-4f61-8ccb-444d4a39af5f","Type":"ContainerStarted","Data":"4f575c935dc6d4804742bef3125c38b00a1fac0d1c4e3c79d8d01c132844166b"} Jul 22 03:33:57.870981 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:57.870935 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-pbb7v" podStartSLOduration=4.828533776 podStartE2EDuration="23.870918345s" totalImagesPullingTime="19.042384569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:57.870293916 +0000 UTC m=+24.663782408" watchObservedRunningTime="2026-07-22 03:33:57.870918345 +0000 UTC m=+24.664406901" Jul 22 03:33:58.739932 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:58.739906 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:33:58.740070 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:58.740033 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:33:58.862351 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:58.862256 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"bb4b24b856a3d7e3f4e83aa15002ae6a2429dac8f118580d28caca0fa852d022"} Jul 22 03:33:59.528475 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:59.528442 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:59.529092 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:59.529071 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:33:59.739692 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:59.739664 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:33:59.739861 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:59.739801 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:33:59.739861 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:59.739836 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:33:59.739986 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:33:59.739963 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:33:59.867293 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:59.867056 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" event={"ID":"a175ab40-db3f-4f61-8ccb-444d4a39af5f","Type":"ContainerStarted","Data":"989ca544769d32c9c6a79e7b21911f73b834eb8b0bf59ccc87d8ec5092f60262"} Jul 22 03:33:59.885703 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:33:59.885647 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-lk6nm" podStartSLOduration=3.886139857 podStartE2EDuration="25.885627888s" totalImagesPullingTime="21.999488031s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:33:59.885134097 +0000 UTC m=+26.678622599" watchObservedRunningTime="2026-07-22 03:33:59.885627888 +0000 UTC m=+26.679116384" Jul 22 03:34:00.739487 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:00.739458 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:00.739649 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:00.739554 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:34:01.740103 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.739877 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:01.740752 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.739933 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:01.740752 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:01.740231 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:34:01.740752 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:01.740256 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:34:01.873210 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.873132 2564 generic.go:356] "Generic (PLEG): container finished" podID="8f61a890-2b9e-46a7-b95d-2fb102461e5f" containerID="0d7896869d65e92dea8167b24d43962d7751d8fda8c7b4a2362ceeaea4a51cec" exitCode=0 Jul 22 03:34:01.873314 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.873209 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerDied","Data":"0d7896869d65e92dea8167b24d43962d7751d8fda8c7b4a2362ceeaea4a51cec"} Jul 22 03:34:01.877484 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.877454 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" event={"ID":"10fe30fd-c5d3-46e9-abb0-b03312fd3174","Type":"ContainerStarted","Data":"4e9f0e118573d81e969ea7df7bb8579703b7102f52a52e62be2a3dfb7ef0fb14"} Jul 22 03:34:01.877734 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.877717 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:34:01.877814 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.877792 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:34:01.893771 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.893753 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:34:01.947494 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:01.947445 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" podStartSLOduration=8.81451018 podStartE2EDuration="27.947434525s" totalImagesPullingTime="19.132924345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:01.945381999 +0000 UTC m=+28.738870492" watchObservedRunningTime="2026-07-22 03:34:01.947434525 +0000 UTC m=+28.740923014" Jul 22 03:34:02.739968 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:02.739942 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:02.740308 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:02.740071 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:34:02.880473 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:02.880447 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:34:02.895096 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:02.895075 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:34:03.301745 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.301722 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k9vqm"] Jul 22 03:34:03.301890 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.301844 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:03.301976 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:03.301954 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:34:03.302406 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.302380 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m79rh"] Jul 22 03:34:03.302520 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.302503 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:03.302626 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:03.302604 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:34:03.303049 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.303031 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gkdws"] Jul 22 03:34:03.303131 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.303118 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:03.303213 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:03.303196 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:34:03.883749 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.883718 2564 generic.go:356] "Generic (PLEG): container finished" podID="8f61a890-2b9e-46a7-b95d-2fb102461e5f" containerID="9893553da497a6f6535c684a03cb9b48bf94d92549fdf0a8ff9a2f7bd3c66beb" exitCode=0 Jul 22 03:34:03.884185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:03.883810 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerDied","Data":"9893553da497a6f6535c684a03cb9b48bf94d92549fdf0a8ff9a2f7bd3c66beb"} Jul 22 03:34:04.740005 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:04.739977 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:04.740180 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:04.740009 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:04.740180 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:04.739977 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:04.740180 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:04.740085 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:34:04.740180 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:04.740165 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:34:04.740414 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:04.740260 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:34:05.671986 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:05.671785 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:34:05.672466 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:05.672077 2564 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 22 03:34:05.672564 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:05.672546 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xxkvv" Jul 22 03:34:05.891111 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:05.891077 2564 generic.go:356] "Generic (PLEG): container finished" podID="8f61a890-2b9e-46a7-b95d-2fb102461e5f" containerID="b9084e7574d88c10bbc5b9c981d7e4cced181d47a59eed673a32c62dd41f1e3f" exitCode=0 Jul 22 03:34:05.891231 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:05.891156 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerDied","Data":"b9084e7574d88c10bbc5b9c981d7e4cced181d47a59eed673a32c62dd41f1e3f"} Jul 22 03:34:06.739003 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:06.738971 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:06.739453 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:06.739389 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:06.739552 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:06.739517 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-gkdws" podUID="2c29c9d9-c861-47a6-bc5d-4e9e316a5c09" Jul 22 03:34:06.739552 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:06.739356 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-k9vqm" podUID="200a8fe2-eab4-4e9f-924f-03db74707208" Jul 22 03:34:06.739726 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:06.739580 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:06.739726 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:06.739700 2564 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-m79rh" podUID="41c2a5d1-2687-40c4-b1dc-869e2f36ee58" Jul 22 03:34:07.342649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:07.342614 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:07.342815 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:07.342680 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:07.342815 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.342764 2564 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:34:07.342815 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.342786 2564 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 03:34:07.342945 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.342840 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs podName:41c2a5d1-2687-40c4-b1dc-869e2f36ee58 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.342819818 +0000 UTC m=+66.136308296 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs") pod "network-metrics-daemon-m79rh" (UID: "41c2a5d1-2687-40c4-b1dc-869e2f36ee58") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 03:34:07.342945 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.342861 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret podName:200a8fe2-eab4-4e9f-924f-03db74707208 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.342850461 +0000 UTC m=+66.136338933 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret") pod "global-pull-secret-syncer-k9vqm" (UID: "200a8fe2-eab4-4e9f-924f-03db74707208") : object "kube-system"/"original-pull-secret" not registered Jul 22 03:34:07.442930 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:07.442907 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:07.443070 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.443045 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 03:34:07.443070 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.443064 2564 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 03:34:07.443070 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.443072 2564 projected.go:196] Error preparing data for projected volume kube-api-access-6tpv9 for pod openshift-network-diagnostics/network-check-target-gkdws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:34:07.443186 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:07.443124 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9 podName:2c29c9d9-c861-47a6-bc5d-4e9e316a5c09 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:39.443107539 +0000 UTC m=+66.236596026 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-6tpv9" (UniqueName: "kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9") pod "network-check-target-gkdws" (UID: "2c29c9d9-c861-47a6-bc5d-4e9e316a5c09") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 03:34:08.497186 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.497160 2564 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-139-52.ec2.internal" event="NodeReady" Jul 22 03:34:08.497730 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.497304 2564 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 03:34:08.554806 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.554724 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-xx9rw"] Jul 22 03:34:08.577534 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.577504 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.602295 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.602263 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 03:34:08.603391 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.603376 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jn6xf\"" Jul 22 03:34:08.608184 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.608166 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tgn8x"] Jul 22 03:34:08.610195 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.610180 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 03:34:08.631324 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.631302 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:08.650202 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.650178 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 03:34:08.653153 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.653134 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8dq25\"" Jul 22 03:34:08.661161 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.661139 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 03:34:08.661861 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.661826 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 03:34:08.662092 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.662068 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xx9rw"] Jul 22 03:34:08.687146 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.687122 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tgn8x"] Jul 22 03:34:08.739611 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.739586 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:08.739611 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.739608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:08.739784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.739583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:08.745383 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.745364 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dzrs8\"" Jul 22 03:34:08.745980 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.745847 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4vks\"" Jul 22 03:34:08.745980 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.745875 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 03:34:08.747481 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.747460 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 03:34:08.751129 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.751112 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 03:34:08.751627 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.751609 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 03:34:08.754662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.754638 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66905f1b-1a5f-49f7-9411-2123829bdb6c-cert\") pod \"ingress-canary-tgn8x\" (UID: \"66905f1b-1a5f-49f7-9411-2123829bdb6c\") " pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:08.754739 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.754728 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.754792 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.754766 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4xrc\" (UniqueName: \"kubernetes.io/projected/66905f1b-1a5f-49f7-9411-2123829bdb6c-kube-api-access-v4xrc\") pod \"ingress-canary-tgn8x\" (UID: \"66905f1b-1a5f-49f7-9411-2123829bdb6c\") " pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:08.754845 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.754791 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/670451d7-67ac-46f2-a8ea-af8b553b7602-tmp-dir\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.754845 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.754819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/670451d7-67ac-46f2-a8ea-af8b553b7602-config-volume\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.754926 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.754908 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbtm9\" (UniqueName: \"kubernetes.io/projected/670451d7-67ac-46f2-a8ea-af8b553b7602-kube-api-access-xbtm9\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.855511 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.855443 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4xrc\" (UniqueName: \"kubernetes.io/projected/66905f1b-1a5f-49f7-9411-2123829bdb6c-kube-api-access-v4xrc\") pod \"ingress-canary-tgn8x\" (UID: \"66905f1b-1a5f-49f7-9411-2123829bdb6c\") " pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:08.855511 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.855480 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/670451d7-67ac-46f2-a8ea-af8b553b7602-tmp-dir\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.855693 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.855591 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/670451d7-67ac-46f2-a8ea-af8b553b7602-config-volume\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.855693 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.855660 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbtm9\" (UniqueName: \"kubernetes.io/projected/670451d7-67ac-46f2-a8ea-af8b553b7602-kube-api-access-xbtm9\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.855799 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.855727 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66905f1b-1a5f-49f7-9411-2123829bdb6c-cert\") pod \"ingress-canary-tgn8x\" (UID: \"66905f1b-1a5f-49f7-9411-2123829bdb6c\") " pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:08.855799 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.855784 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.855799 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.855794 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/670451d7-67ac-46f2-a8ea-af8b553b7602-tmp-dir\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.855946 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:08.855889 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:34:08.855946 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:08.855943 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls podName:670451d7-67ac-46f2-a8ea-af8b553b7602 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:09.35592464 +0000 UTC m=+36.149413125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls") pod "dns-default-xx9rw" (UID: "670451d7-67ac-46f2-a8ea-af8b553b7602") : secret "dns-default-metrics-tls" not found Jul 22 03:34:08.856166 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.856147 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/670451d7-67ac-46f2-a8ea-af8b553b7602-config-volume\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.860610 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.860591 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66905f1b-1a5f-49f7-9411-2123829bdb6c-cert\") pod \"ingress-canary-tgn8x\" (UID: \"66905f1b-1a5f-49f7-9411-2123829bdb6c\") " pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:08.864082 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.864060 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg"] Jul 22 03:34:08.870118 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.870096 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbtm9\" (UniqueName: \"kubernetes.io/projected/670451d7-67ac-46f2-a8ea-af8b553b7602-kube-api-access-xbtm9\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:08.870549 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.870529 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4xrc\" (UniqueName: \"kubernetes.io/projected/66905f1b-1a5f-49f7-9411-2123829bdb6c-kube-api-access-v4xrc\") pod \"ingress-canary-tgn8x\" (UID: \"66905f1b-1a5f-49f7-9411-2123829bdb6c\") " pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:08.893516 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.893488 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg"] Jul 22 03:34:08.893618 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.893605 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" Jul 22 03:34:08.896734 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.896712 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-f56kj\"" Jul 22 03:34:08.940218 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:08.940198 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tgn8x" Jul 22 03:34:09.057782 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:09.057697 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pj9cm\" (UniqueName: \"kubernetes.io/projected/67e9766f-3579-442c-9558-6152d83d7ca5-kube-api-access-pj9cm\") pod \"network-check-source-7bc5b6bf85-2ktbg\" (UID: \"67e9766f-3579-442c-9558-6152d83d7ca5\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" Jul 22 03:34:09.158170 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:09.158093 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pj9cm\" (UniqueName: \"kubernetes.io/projected/67e9766f-3579-442c-9558-6152d83d7ca5-kube-api-access-pj9cm\") pod \"network-check-source-7bc5b6bf85-2ktbg\" (UID: \"67e9766f-3579-442c-9558-6152d83d7ca5\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" Jul 22 03:34:09.169280 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:09.169256 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pj9cm\" (UniqueName: \"kubernetes.io/projected/67e9766f-3579-442c-9558-6152d83d7ca5-kube-api-access-pj9cm\") pod \"network-check-source-7bc5b6bf85-2ktbg\" (UID: \"67e9766f-3579-442c-9558-6152d83d7ca5\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" Jul 22 03:34:09.204149 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:09.204127 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" Jul 22 03:34:09.360266 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:09.360237 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:09.360468 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:09.360394 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:34:09.360468 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:09.360453 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls podName:670451d7-67ac-46f2-a8ea-af8b553b7602 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:10.360433552 +0000 UTC m=+37.153922023 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls") pod "dns-default-xx9rw" (UID: "670451d7-67ac-46f2-a8ea-af8b553b7602") : secret "dns-default-metrics-tls" not found Jul 22 03:34:10.087729 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:10.087568 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tgn8x"] Jul 22 03:34:10.091180 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:10.091141 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66905f1b_1a5f_49f7_9411_2123829bdb6c.slice/crio-b632117bb5f2cae41291366600f3bf6f92b09f3189a0c80584455a07333d0245 WatchSource:0}: Error finding container b632117bb5f2cae41291366600f3bf6f92b09f3189a0c80584455a07333d0245: Status 404 returned error can't find the container with id b632117bb5f2cae41291366600f3bf6f92b09f3189a0c80584455a07333d0245 Jul 22 03:34:10.091623 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:10.091602 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg"] Jul 22 03:34:10.095081 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:10.095057 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod67e9766f_3579_442c_9558_6152d83d7ca5.slice/crio-05ee790863d9a1a632b333aa8d9a6f9b36e6bc7470471bfc47273b8eefc4ee61 WatchSource:0}: Error finding container 05ee790863d9a1a632b333aa8d9a6f9b36e6bc7470471bfc47273b8eefc4ee61: Status 404 returned error can't find the container with id 05ee790863d9a1a632b333aa8d9a6f9b36e6bc7470471bfc47273b8eefc4ee61 Jul 22 03:34:10.368368 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:10.368326 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:10.368459 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:10.368441 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:34:10.368504 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:10.368496 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls podName:670451d7-67ac-46f2-a8ea-af8b553b7602 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:12.368482477 +0000 UTC m=+39.161970973 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls") pod "dns-default-xx9rw" (UID: "670451d7-67ac-46f2-a8ea-af8b553b7602") : secret "dns-default-metrics-tls" not found Jul 22 03:34:10.906258 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:10.906226 2564 generic.go:356] "Generic (PLEG): container finished" podID="8f61a890-2b9e-46a7-b95d-2fb102461e5f" containerID="6c6f1a58e6ae4af92d9ae07ead8096085a1e08024a4b4ce7d8236df5565508e5" exitCode=0 Jul 22 03:34:10.906454 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:10.906351 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerDied","Data":"6c6f1a58e6ae4af92d9ae07ead8096085a1e08024a4b4ce7d8236df5565508e5"} Jul 22 03:34:10.908009 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:10.907968 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tgn8x" event={"ID":"66905f1b-1a5f-49f7-9411-2123829bdb6c","Type":"ContainerStarted","Data":"b632117bb5f2cae41291366600f3bf6f92b09f3189a0c80584455a07333d0245"} Jul 22 03:34:10.909877 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:10.909630 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" event={"ID":"67e9766f-3579-442c-9558-6152d83d7ca5","Type":"ContainerStarted","Data":"05ee790863d9a1a632b333aa8d9a6f9b36e6bc7470471bfc47273b8eefc4ee61"} Jul 22 03:34:11.916211 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:11.916173 2564 generic.go:356] "Generic (PLEG): container finished" podID="8f61a890-2b9e-46a7-b95d-2fb102461e5f" containerID="4327f1012ae289b38157e4fc37494b57ebce48d12eac322f257c6257b3761e14" exitCode=0 Jul 22 03:34:11.916680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:11.916245 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerDied","Data":"4327f1012ae289b38157e4fc37494b57ebce48d12eac322f257c6257b3761e14"} Jul 22 03:34:12.385020 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:12.384969 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:12.385191 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:12.385116 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:34:12.385191 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:12.385191 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls podName:670451d7-67ac-46f2-a8ea-af8b553b7602 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:16.385170362 +0000 UTC m=+43.178658832 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls") pod "dns-default-xx9rw" (UID: "670451d7-67ac-46f2-a8ea-af8b553b7602") : secret "dns-default-metrics-tls" not found Jul 22 03:34:13.925312 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:13.925204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7wddc" event={"ID":"8f61a890-2b9e-46a7-b95d-2fb102461e5f","Type":"ContainerStarted","Data":"0da2714fe90d21b0ad6052b1324faa027231f5e5c9a0b9883b1ce99d78af156c"} Jul 22 03:34:13.926838 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:13.926803 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tgn8x" event={"ID":"66905f1b-1a5f-49f7-9411-2123829bdb6c","Type":"ContainerStarted","Data":"5f29d5e6cee3812bab83a9a54c30e92ecbf5005aa56e589f5df239835e89f926"} Jul 22 03:34:13.928320 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:13.928295 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" event={"ID":"67e9766f-3579-442c-9558-6152d83d7ca5","Type":"ContainerStarted","Data":"0f15ce3e076731f358d639e18e4dbf32cb3eec897a25f057709a51349cbc7f25"} Jul 22 03:34:13.955605 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:13.955564 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7wddc" podStartSLOduration=9.67821654 podStartE2EDuration="39.955547822s" totalImagesPullingTime="30.277331282s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:13.954799011 +0000 UTC m=+40.748287502" watchObservedRunningTime="2026-07-22 03:34:13.955547822 +0000 UTC m=+40.749036314" Jul 22 03:34:13.971735 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:13.971701 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-2ktbg" podStartSLOduration=2.336689976 podStartE2EDuration="5.971688231s" totalImagesPullingTime="3.634998255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:13.971250168 +0000 UTC m=+40.764738661" watchObservedRunningTime="2026-07-22 03:34:13.971688231 +0000 UTC m=+40.765176722" Jul 22 03:34:13.987121 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:13.987084 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tgn8x" podStartSLOduration=2.357141065 podStartE2EDuration="5.98707226s" totalImagesPullingTime="3.629931195s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:13.986389822 +0000 UTC m=+40.779878314" watchObservedRunningTime="2026-07-22 03:34:13.98707226 +0000 UTC m=+40.780560751" Jul 22 03:34:16.410245 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:16.410210 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:16.410638 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:16.410372 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:34:16.410638 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:16.410427 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls podName:670451d7-67ac-46f2-a8ea-af8b553b7602 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:24.410412147 +0000 UTC m=+51.203900616 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls") pod "dns-default-xx9rw" (UID: "670451d7-67ac-46f2-a8ea-af8b553b7602") : secret "dns-default-metrics-tls" not found Jul 22 03:34:17.215547 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.215514 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8"] Jul 22 03:34:17.247600 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.247579 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8"] Jul 22 03:34:17.247719 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.247669 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" Jul 22 03:34:17.250880 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.250857 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 22 03:34:17.252290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.252269 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-m8g9l\"" Jul 22 03:34:17.252400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.252269 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 22 03:34:17.317527 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.317504 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zml5b\" (UniqueName: \"kubernetes.io/projected/2d0f7975-a0a0-4ab5-936d-8025af234065-kube-api-access-zml5b\") pod \"migrator-7cc9c7ffd7-pmzq8\" (UID: \"2d0f7975-a0a0-4ab5-936d-8025af234065\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" Jul 22 03:34:17.407076 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.407047 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-dxbf7"] Jul 22 03:34:17.418212 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.418185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zml5b\" (UniqueName: \"kubernetes.io/projected/2d0f7975-a0a0-4ab5-936d-8025af234065-kube-api-access-zml5b\") pod \"migrator-7cc9c7ffd7-pmzq8\" (UID: \"2d0f7975-a0a0-4ab5-936d-8025af234065\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" Jul 22 03:34:17.428241 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.428220 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zml5b\" (UniqueName: \"kubernetes.io/projected/2d0f7975-a0a0-4ab5-936d-8025af234065-kube-api-access-zml5b\") pod \"migrator-7cc9c7ffd7-pmzq8\" (UID: \"2d0f7975-a0a0-4ab5-936d-8025af234065\") " pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" Jul 22 03:34:17.429422 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.429398 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dxbf7"] Jul 22 03:34:17.429544 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.429531 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.432363 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.432324 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 03:34:17.432629 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.432602 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 03:34:17.432729 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.432611 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 03:34:17.432729 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.432638 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cwd77\"" Jul 22 03:34:17.432729 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.432680 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 03:34:17.519231 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.519213 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlmbs\" (UniqueName: \"kubernetes.io/projected/79027f34-2de1-4e27-9422-43081303a8d8-kube-api-access-vlmbs\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.519355 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.519241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.519355 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.519273 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79027f34-2de1-4e27-9422-43081303a8d8-data-volume\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.519448 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.519394 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79027f34-2de1-4e27-9422-43081303a8d8-crio-socket\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.519448 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.519432 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.556423 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.556402 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" Jul 22 03:34:17.620253 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.620224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79027f34-2de1-4e27-9422-43081303a8d8-crio-socket\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.620425 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.620268 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.620425 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.620297 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vlmbs\" (UniqueName: \"kubernetes.io/projected/79027f34-2de1-4e27-9422-43081303a8d8-kube-api-access-vlmbs\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.620425 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.620319 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.620425 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:17.620409 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:34:17.620425 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:17.620410 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:34:17.620686 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.620441 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/79027f34-2de1-4e27-9422-43081303a8d8-crio-socket\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.620686 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:17.620476 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:18.120458998 +0000 UTC m=+44.913947506 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : configmap "kube-rbac-proxy" not found Jul 22 03:34:17.620686 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:17.620496 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:18.120484333 +0000 UTC m=+44.913972806 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : secret "insights-runtime-extractor-tls" not found Jul 22 03:34:17.620686 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.620460 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79027f34-2de1-4e27-9422-43081303a8d8-data-volume\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.620896 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.620753 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/79027f34-2de1-4e27-9422-43081303a8d8-data-volume\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.630815 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.630672 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlmbs\" (UniqueName: \"kubernetes.io/projected/79027f34-2de1-4e27-9422-43081303a8d8-kube-api-access-vlmbs\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:17.721399 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.721375 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8"] Jul 22 03:34:17.723966 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:17.723945 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d0f7975_a0a0_4ab5_936d_8025af234065.slice/crio-2adf58d92bcdd298a87239fef3625b40b3fbf0a38b1b5ca4ed72fe299cc65880 WatchSource:0}: Error finding container 2adf58d92bcdd298a87239fef3625b40b3fbf0a38b1b5ca4ed72fe299cc65880: Status 404 returned error can't find the container with id 2adf58d92bcdd298a87239fef3625b40b3fbf0a38b1b5ca4ed72fe299cc65880 Jul 22 03:34:17.938544 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:17.938476 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" event={"ID":"2d0f7975-a0a0-4ab5-936d-8025af234065","Type":"ContainerStarted","Data":"2adf58d92bcdd298a87239fef3625b40b3fbf0a38b1b5ca4ed72fe299cc65880"} Jul 22 03:34:18.123521 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.123485 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:18.123650 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.123532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:18.123650 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:18.123617 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:34:18.123740 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:18.123646 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:34:18.123740 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:18.123672 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:19.123654698 +0000 UTC m=+45.917143168 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : configmap "kube-rbac-proxy" not found Jul 22 03:34:18.123740 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:18.123705 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:19.123690577 +0000 UTC m=+45.917179047 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : secret "insights-runtime-extractor-tls" not found Jul 22 03:34:18.683549 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.683515 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-58598695f7-v9mxr"] Jul 22 03:34:18.706623 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.706566 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-v9mxr"] Jul 22 03:34:18.706759 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.706693 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.711597 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.711398 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 22 03:34:18.711895 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.711877 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-gx7jr\"" Jul 22 03:34:18.712582 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.712556 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 22 03:34:18.713560 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.713535 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 22 03:34:18.713674 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.713585 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 22 03:34:18.727445 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.727424 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-signing-cabundle\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.727572 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.727497 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-signing-key\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.727637 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.727581 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2jhg\" (UniqueName: \"kubernetes.io/projected/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-kube-api-access-g2jhg\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.828591 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.828564 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-signing-key\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.828703 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.828603 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2jhg\" (UniqueName: \"kubernetes.io/projected/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-kube-api-access-g2jhg\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.828703 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.828689 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-signing-cabundle\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.829477 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.829456 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-signing-cabundle\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.831500 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.831472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-signing-key\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:18.838320 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:18.838293 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2jhg\" (UniqueName: \"kubernetes.io/projected/5a4ddc78-0211-43f7-b35d-c42b6e9e25c0-kube-api-access-g2jhg\") pod \"service-ca-58598695f7-v9mxr\" (UID: \"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0\") " pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:19.017683 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.017646 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-58598695f7-v9mxr" Jul 22 03:34:19.130809 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.130777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:19.130957 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.130840 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:19.131014 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:19.130962 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:34:19.131065 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:19.131024 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:21.13100472 +0000 UTC m=+47.924493214 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : configmap "kube-rbac-proxy" not found Jul 22 03:34:19.131483 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:19.131462 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:34:19.131595 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:19.131522 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:21.131504982 +0000 UTC m=+47.924993473 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : secret "insights-runtime-extractor-tls" not found Jul 22 03:34:19.326469 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.326447 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-58598695f7-v9mxr"] Jul 22 03:34:19.328458 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:19.328427 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a4ddc78_0211_43f7_b35d_c42b6e9e25c0.slice/crio-7b0ce48063cbfa3f11b3943d7ed07a582641823203255f4a45838651c791cfe7 WatchSource:0}: Error finding container 7b0ce48063cbfa3f11b3943d7ed07a582641823203255f4a45838651c791cfe7: Status 404 returned error can't find the container with id 7b0ce48063cbfa3f11b3943d7ed07a582641823203255f4a45838651c791cfe7 Jul 22 03:34:19.681023 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.681002 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fqjtc_7fca8f59-2e5c-403a-adba-6d628d831737/dns-node-resolver/0.log" Jul 22 03:34:19.944941 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.944876 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-v9mxr" event={"ID":"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0","Type":"ContainerStarted","Data":"7b0ce48063cbfa3f11b3943d7ed07a582641823203255f4a45838651c791cfe7"} Jul 22 03:34:19.946744 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.946717 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" event={"ID":"2d0f7975-a0a0-4ab5-936d-8025af234065","Type":"ContainerStarted","Data":"caa4709af7700b07f7e8f530c5b6fa7c054ccd5339093e8efb33348b735a4f08"} Jul 22 03:34:19.946877 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.946760 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" event={"ID":"2d0f7975-a0a0-4ab5-936d-8025af234065","Type":"ContainerStarted","Data":"7cc602f2401e0bf95b1a17ee67326858715a744e0eb73d97f2f6a5b2d044c4d4"} Jul 22 03:34:19.968349 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:19.968276 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7cc9c7ffd7-pmzq8" podStartSLOduration=1.181990586 podStartE2EDuration="2.968255767s" totalImagesPullingTime="1.786265181s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:19.968092914 +0000 UTC m=+46.761581407" watchObservedRunningTime="2026-07-22 03:34:19.968255767 +0000 UTC m=+46.761744260" Jul 22 03:34:20.476326 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:20.476301 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4pgwk_db43e671-0bc5-403f-be13-99333644b038/node-ca/0.log" Jul 22 03:34:21.145831 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:21.145798 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:21.145831 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:21.145842 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:21.146323 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:21.145918 2564 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 03:34:21.146323 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:21.145969 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:34:21.146323 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:21.145986 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:25.145968455 +0000 UTC m=+51.939456970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : configmap "kube-rbac-proxy" not found Jul 22 03:34:21.146323 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:21.146019 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:25.146005518 +0000 UTC m=+51.939493990 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : secret "insights-runtime-extractor-tls" not found Jul 22 03:34:21.476814 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:21.476751 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tgn8x_66905f1b-1a5f-49f7-9411-2123829bdb6c/serve-healthcheck-canary/0.log" Jul 22 03:34:21.676154 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:21.676125 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pmzq8_2d0f7975-a0a0-4ab5-936d-8025af234065/migrator/0.log" Jul 22 03:34:21.879741 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:21.879710 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pmzq8_2d0f7975-a0a0-4ab5-936d-8025af234065/graceful-termination/0.log" Jul 22 03:34:21.953867 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:21.953841 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-58598695f7-v9mxr" event={"ID":"5a4ddc78-0211-43f7-b35d-c42b6e9e25c0","Type":"ContainerStarted","Data":"04be7644e622077f1d791c0e78a54545c341249a21cf77dbf3ad28a4550402c6"} Jul 22 03:34:21.973000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:21.972962 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-58598695f7-v9mxr" podStartSLOduration=1.6560942490000001 podStartE2EDuration="3.972948267s" totalImagesPullingTime="2.316854018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:21.971763429 +0000 UTC m=+48.765251918" watchObservedRunningTime="2026-07-22 03:34:21.972948267 +0000 UTC m=+48.766436753" Jul 22 03:34:24.468780 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:24.468745 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:24.469215 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:24.468860 2564 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 03:34:24.469215 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:24.468919 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls podName:670451d7-67ac-46f2-a8ea-af8b553b7602 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:40.468904494 +0000 UTC m=+67.262392966 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls") pod "dns-default-xx9rw" (UID: "670451d7-67ac-46f2-a8ea-af8b553b7602") : secret "dns-default-metrics-tls" not found Jul 22 03:34:25.174259 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:25.174224 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:25.174539 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:25.174272 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:25.174539 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:25.174395 2564 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 03:34:25.174539 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:34:25.174457 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls podName:79027f34-2de1-4e27-9422-43081303a8d8 nodeName:}" failed. No retries permitted until 2026-07-22 03:34:33.174439486 +0000 UTC m=+59.967927970 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls") pod "insights-runtime-extractor-dxbf7" (UID: "79027f34-2de1-4e27-9422-43081303a8d8") : secret "insights-runtime-extractor-tls" not found Jul 22 03:34:25.174842 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:25.174825 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/79027f34-2de1-4e27-9422-43081303a8d8-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:33.229940 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:33.229903 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:33.232312 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:33.232289 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/79027f34-2de1-4e27-9422-43081303a8d8-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-dxbf7\" (UID: \"79027f34-2de1-4e27-9422-43081303a8d8\") " pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:33.340605 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:33.340583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-dxbf7" Jul 22 03:34:33.460876 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:33.460847 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-dxbf7"] Jul 22 03:34:33.464100 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:33.464077 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79027f34_2de1_4e27_9422_43081303a8d8.slice/crio-a1db2c6f0be7ae2967f594d97367eb3f5dfd40d802791c160f867db05631122f WatchSource:0}: Error finding container a1db2c6f0be7ae2967f594d97367eb3f5dfd40d802791c160f867db05631122f: Status 404 returned error can't find the container with id a1db2c6f0be7ae2967f594d97367eb3f5dfd40d802791c160f867db05631122f Jul 22 03:34:33.986032 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:33.985953 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dxbf7" event={"ID":"79027f34-2de1-4e27-9422-43081303a8d8","Type":"ContainerStarted","Data":"a1db2c6f0be7ae2967f594d97367eb3f5dfd40d802791c160f867db05631122f"} Jul 22 03:34:34.897763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:34.897737 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5pkln" Jul 22 03:34:39.003790 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.003737 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dxbf7" event={"ID":"79027f34-2de1-4e27-9422-43081303a8d8","Type":"ContainerStarted","Data":"2b381dfc26e97aa911f0d4d3e4444da227aace3a40161fce79d934948b3deffc"} Jul 22 03:34:39.375317 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.375239 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:39.375317 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.375306 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:39.378571 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.378552 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 03:34:39.379694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.379674 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 03:34:39.388302 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.388276 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/41c2a5d1-2687-40c4-b1dc-869e2f36ee58-metrics-certs\") pod \"network-metrics-daemon-m79rh\" (UID: \"41c2a5d1-2687-40c4-b1dc-869e2f36ee58\") " pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:39.388498 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.388481 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/200a8fe2-eab4-4e9f-924f-03db74707208-original-pull-secret\") pod \"global-pull-secret-syncer-k9vqm\" (UID: \"200a8fe2-eab4-4e9f-924f-03db74707208\") " pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:39.475788 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.475763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:39.478296 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.478275 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tpv9\" (UniqueName: \"kubernetes.io/projected/2c29c9d9-c861-47a6-bc5d-4e9e316a5c09-kube-api-access-6tpv9\") pod \"network-check-target-gkdws\" (UID: \"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09\") " pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:39.653948 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.653880 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-h4vks\"" Jul 22 03:34:39.657405 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.657385 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-k9vqm" Jul 22 03:34:39.661572 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.661551 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:39.664786 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.664766 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-dzrs8\"" Jul 22 03:34:39.672602 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:39.672583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-m79rh" Jul 22 03:34:40.424264 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.424229 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl"] Jul 22 03:34:40.437534 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.437505 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h"] Jul 22 03:34:40.437681 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.437650 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.439669 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.439648 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.446957 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.446932 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl"] Jul 22 03:34:40.447626 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.447608 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh"] Jul 22 03:34:40.450632 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.450614 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.454712 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.454514 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 22 03:34:40.455164 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.455141 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 03:34:40.455257 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.455199 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 03:34:40.455354 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.455145 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 22 03:34:40.455711 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.455695 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 03:34:40.455807 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.455795 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-zvlwb\"" Jul 22 03:34:40.455922 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.455905 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 03:34:40.456509 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.456492 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-8rgv8\"" Jul 22 03:34:40.461857 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.461839 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 22 03:34:40.471927 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.471906 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h"] Jul 22 03:34:40.472797 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.472748 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh"] Jul 22 03:34:40.482557 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.482535 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:40.482644 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.482601 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e5411e49-cf4d-4c68-96c0-b0d301c58695-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-5phsl\" (UID: \"e5411e49-cf4d-4c68-96c0-b0d301c58695\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.482644 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.482630 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79981694-7262-47fa-9b35-f1b8bb1f0410-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b7bd688cd-klt5h\" (UID: \"79981694-7262-47fa-9b35-f1b8bb1f0410\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.482749 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.482721 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5411e49-cf4d-4c68-96c0-b0d301c58695-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5phsl\" (UID: \"e5411e49-cf4d-4c68-96c0-b0d301c58695\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.482803 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.482773 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4mlf\" (UniqueName: \"kubernetes.io/projected/79981694-7262-47fa-9b35-f1b8bb1f0410-kube-api-access-n4mlf\") pod \"managed-serviceaccount-addon-agent-7b7bd688cd-klt5h\" (UID: \"79981694-7262-47fa-9b35-f1b8bb1f0410\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.485152 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.485134 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/670451d7-67ac-46f2-a8ea-af8b553b7602-metrics-tls\") pod \"dns-default-xx9rw\" (UID: \"670451d7-67ac-46f2-a8ea-af8b553b7602\") " pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:40.583544 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.583511 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04756259-81e3-448e-9544-8c71ab94d644-tmp\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.583706 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.583577 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5411e49-cf4d-4c68-96c0-b0d301c58695-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5phsl\" (UID: \"e5411e49-cf4d-4c68-96c0-b0d301c58695\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.583706 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.583602 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4jdf\" (UniqueName: \"kubernetes.io/projected/04756259-81e3-448e-9544-8c71ab94d644-kube-api-access-z4jdf\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.583706 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.583627 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4mlf\" (UniqueName: \"kubernetes.io/projected/79981694-7262-47fa-9b35-f1b8bb1f0410-kube-api-access-n4mlf\") pod \"managed-serviceaccount-addon-agent-7b7bd688cd-klt5h\" (UID: \"79981694-7262-47fa-9b35-f1b8bb1f0410\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.583706 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.583661 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e5411e49-cf4d-4c68-96c0-b0d301c58695-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-5phsl\" (UID: \"e5411e49-cf4d-4c68-96c0-b0d301c58695\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.583876 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.583706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/04756259-81e3-448e-9544-8c71ab94d644-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.583876 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.583793 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79981694-7262-47fa-9b35-f1b8bb1f0410-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b7bd688cd-klt5h\" (UID: \"79981694-7262-47fa-9b35-f1b8bb1f0410\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.584405 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.584348 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e5411e49-cf4d-4c68-96c0-b0d301c58695-nginx-conf\") pod \"networking-console-plugin-7bc7489cb5-5phsl\" (UID: \"e5411e49-cf4d-4c68-96c0-b0d301c58695\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.586381 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.586354 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79981694-7262-47fa-9b35-f1b8bb1f0410-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-7b7bd688cd-klt5h\" (UID: \"79981694-7262-47fa-9b35-f1b8bb1f0410\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.586381 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.586359 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5411e49-cf4d-4c68-96c0-b0d301c58695-networking-console-plugin-cert\") pod \"networking-console-plugin-7bc7489cb5-5phsl\" (UID: \"e5411e49-cf4d-4c68-96c0-b0d301c58695\") " pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.602466 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.602439 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4mlf\" (UniqueName: \"kubernetes.io/projected/79981694-7262-47fa-9b35-f1b8bb1f0410-kube-api-access-n4mlf\") pod \"managed-serviceaccount-addon-agent-7b7bd688cd-klt5h\" (UID: \"79981694-7262-47fa-9b35-f1b8bb1f0410\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.685030 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.684967 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/04756259-81e3-448e-9544-8c71ab94d644-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.685030 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.685007 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04756259-81e3-448e-9544-8c71ab94d644-tmp\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.685242 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.685037 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4jdf\" (UniqueName: \"kubernetes.io/projected/04756259-81e3-448e-9544-8c71ab94d644-kube-api-access-z4jdf\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.685416 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.685397 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04756259-81e3-448e-9544-8c71ab94d644-tmp\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.687512 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.687488 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/04756259-81e3-448e-9544-8c71ab94d644-klusterlet-config\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.689744 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.689723 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jn6xf\"" Jul 22 03:34:40.697856 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.697838 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:40.701784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.701763 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4jdf\" (UniqueName: \"kubernetes.io/projected/04756259-81e3-448e-9544-8c71ab94d644-kube-api-access-z4jdf\") pod \"klusterlet-addon-workmgr-74d568876b-glsjh\" (UID: \"04756259-81e3-448e-9544-8c71ab94d644\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:40.750354 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.750306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" Jul 22 03:34:40.772426 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.772406 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" Jul 22 03:34:40.778300 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:40.778281 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:41.635885 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:41.635856 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl"] Jul 22 03:34:41.638270 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:41.638233 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5411e49_cf4d_4c68_96c0_b0d301c58695.slice/crio-7874f8f65526c66ddfeff1192b9bb456fb3acbd9a1918dd6bded2e560da76ec3 WatchSource:0}: Error finding container 7874f8f65526c66ddfeff1192b9bb456fb3acbd9a1918dd6bded2e560da76ec3: Status 404 returned error can't find the container with id 7874f8f65526c66ddfeff1192b9bb456fb3acbd9a1918dd6bded2e560da76ec3 Jul 22 03:34:41.733256 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:41.732989 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-m79rh"] Jul 22 03:34:41.735168 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:41.735144 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41c2a5d1_2687_40c4_b1dc_869e2f36ee58.slice/crio-87ccb88f120f010b877cf6bb7925fb4861a15fce58dff1a951adab4dbb9c5d3f WatchSource:0}: Error finding container 87ccb88f120f010b877cf6bb7925fb4861a15fce58dff1a951adab4dbb9c5d3f: Status 404 returned error can't find the container with id 87ccb88f120f010b877cf6bb7925fb4861a15fce58dff1a951adab4dbb9c5d3f Jul 22 03:34:41.775594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:41.775530 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-k9vqm"] Jul 22 03:34:41.778885 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:41.778864 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-gkdws"] Jul 22 03:34:41.782055 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:41.782030 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c29c9d9_c861_47a6_bc5d_4e9e316a5c09.slice/crio-d1e83683669d9f4262a722f651165b509c7b721a0d7c18edb3409de6742a5879 WatchSource:0}: Error finding container d1e83683669d9f4262a722f651165b509c7b721a0d7c18edb3409de6742a5879: Status 404 returned error can't find the container with id d1e83683669d9f4262a722f651165b509c7b721a0d7c18edb3409de6742a5879 Jul 22 03:34:41.782787 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:41.782722 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-xx9rw"] Jul 22 03:34:41.786391 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:41.786369 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod670451d7_67ac_46f2_a8ea_af8b553b7602.slice/crio-7fc082e1a293b2bc716e23446a8907caf459fbd07bd4e4d92a78a5618250ad74 WatchSource:0}: Error finding container 7fc082e1a293b2bc716e23446a8907caf459fbd07bd4e4d92a78a5618250ad74: Status 404 returned error can't find the container with id 7fc082e1a293b2bc716e23446a8907caf459fbd07bd4e4d92a78a5618250ad74 Jul 22 03:34:41.816730 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:41.816705 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h"] Jul 22 03:34:41.817790 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:41.817771 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh"] Jul 22 03:34:41.819025 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:41.819005 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod79981694_7262_47fa_9b35_f1b8bb1f0410.slice/crio-516360d6fcf06a53af0af838797985a367b5e786b63844f799ceeeebc94e4d44 WatchSource:0}: Error finding container 516360d6fcf06a53af0af838797985a367b5e786b63844f799ceeeebc94e4d44: Status 404 returned error can't find the container with id 516360d6fcf06a53af0af838797985a367b5e786b63844f799ceeeebc94e4d44 Jul 22 03:34:41.819864 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:41.819845 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04756259_81e3_448e_9544_8c71ab94d644.slice/crio-656664e3d5710f08499fb26fd8ae28fc0cff08dc89a1b3afaca36cb16abf3f14 WatchSource:0}: Error finding container 656664e3d5710f08499fb26fd8ae28fc0cff08dc89a1b3afaca36cb16abf3f14: Status 404 returned error can't find the container with id 656664e3d5710f08499fb26fd8ae28fc0cff08dc89a1b3afaca36cb16abf3f14 Jul 22 03:34:42.015761 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.015725 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" event={"ID":"04756259-81e3-448e-9544-8c71ab94d644","Type":"ContainerStarted","Data":"656664e3d5710f08499fb26fd8ae28fc0cff08dc89a1b3afaca36cb16abf3f14"} Jul 22 03:34:42.017031 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.016991 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" event={"ID":"79981694-7262-47fa-9b35-f1b8bb1f0410","Type":"ContainerStarted","Data":"516360d6fcf06a53af0af838797985a367b5e786b63844f799ceeeebc94e4d44"} Jul 22 03:34:42.018180 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.018146 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" event={"ID":"e5411e49-cf4d-4c68-96c0-b0d301c58695","Type":"ContainerStarted","Data":"7874f8f65526c66ddfeff1192b9bb456fb3acbd9a1918dd6bded2e560da76ec3"} Jul 22 03:34:42.019924 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.019903 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gkdws" event={"ID":"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09","Type":"ContainerStarted","Data":"f9322d3939141b74dd1c65c89b64a12baf7f8a0884031686d66d58221db1e9fe"} Jul 22 03:34:42.020037 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.019943 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-gkdws" event={"ID":"2c29c9d9-c861-47a6-bc5d-4e9e316a5c09","Type":"ContainerStarted","Data":"d1e83683669d9f4262a722f651165b509c7b721a0d7c18edb3409de6742a5879"} Jul 22 03:34:42.020098 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.020036 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:34:42.021383 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.021363 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k9vqm" event={"ID":"200a8fe2-eab4-4e9f-924f-03db74707208","Type":"ContainerStarted","Data":"21a8c47116ef741c82c6ab1639322d957fd5242e9f95386f33fdee60555f023d"} Jul 22 03:34:42.023226 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.023202 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dxbf7" event={"ID":"79027f34-2de1-4e27-9422-43081303a8d8","Type":"ContainerStarted","Data":"600072be14f8838e5ab501245bf1657301b9edb78f3664adde6cbe41c441a1fb"} Jul 22 03:34:42.024495 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.024464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m79rh" event={"ID":"41c2a5d1-2687-40c4-b1dc-869e2f36ee58","Type":"ContainerStarted","Data":"87ccb88f120f010b877cf6bb7925fb4861a15fce58dff1a951adab4dbb9c5d3f"} Jul 22 03:34:42.025602 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.025577 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx9rw" event={"ID":"670451d7-67ac-46f2-a8ea-af8b553b7602","Type":"ContainerStarted","Data":"7fc082e1a293b2bc716e23446a8907caf459fbd07bd4e4d92a78a5618250ad74"} Jul 22 03:34:42.037346 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:42.037273 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-gkdws" podStartSLOduration=68.037257523 podStartE2EDuration="1m8.037257523s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:42.035866285 +0000 UTC m=+68.829354775" watchObservedRunningTime="2026-07-22 03:34:42.037257523 +0000 UTC m=+68.830746016" Jul 22 03:34:44.497131 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.497097 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64f48b846d-8n5kd"] Jul 22 03:34:44.516252 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.516220 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.518349 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.518288 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f48b846d-8n5kd"] Jul 22 03:34:44.519185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.519153 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 03:34:44.520108 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.519556 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 03:34:44.520108 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.519764 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 03:34:44.520108 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.519967 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 03:34:44.522439 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.521653 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 03:34:44.522439 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.521889 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nq27z\"" Jul 22 03:34:44.522439 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.522061 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 03:34:44.522439 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.522275 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 03:34:44.618286 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.617940 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-oauth-config\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.618286 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.617986 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpw94\" (UniqueName: \"kubernetes.io/projected/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-kube-api-access-vpw94\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.618286 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.618024 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-serving-cert\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.618286 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.618061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-config\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.618286 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.618087 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-service-ca\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.618286 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.618119 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-oauth-serving-cert\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.718884 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.718847 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-config\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.719038 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.718891 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-service-ca\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.719038 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.718921 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-oauth-serving-cert\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.719038 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.718973 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-oauth-config\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.719038 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.719000 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpw94\" (UniqueName: \"kubernetes.io/projected/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-kube-api-access-vpw94\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.719234 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.719039 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-serving-cert\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.719945 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.719917 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-oauth-serving-cert\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.720234 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.720187 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-config\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.720373 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.720281 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-service-ca\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.723031 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.723008 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-serving-cert\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.726526 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.726503 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-oauth-config\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.732259 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.732235 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpw94\" (UniqueName: \"kubernetes.io/projected/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-kube-api-access-vpw94\") pod \"console-64f48b846d-8n5kd\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:44.830928 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:44.830604 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:50.849133 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:50.844969 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d"] Jul 22 03:34:50.858230 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:50.858188 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d"] Jul 22 03:34:50.858354 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:50.858315 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" Jul 22 03:34:50.866447 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:50.866388 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 22 03:34:50.867487 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:50.867400 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-pvlgk\"" Jul 22 03:34:50.903860 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:50.903828 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f48b846d-8n5kd"] Jul 22 03:34:50.914499 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:50.914474 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82935d60_0991_4b2a_8bd0_ad0fabe57fe1.slice/crio-0fb838a709d7e6c9475016ffb36e1d5f914324882d8ef6f911456c4b56fc36b5 WatchSource:0}: Error finding container 0fb838a709d7e6c9475016ffb36e1d5f914324882d8ef6f911456c4b56fc36b5: Status 404 returned error can't find the container with id 0fb838a709d7e6c9475016ffb36e1d5f914324882d8ef6f911456c4b56fc36b5 Jul 22 03:34:50.965539 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:50.965312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-h2p8d\" (UID: \"4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" Jul 22 03:34:51.065605 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.064929 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f48b846d-8n5kd" event={"ID":"82935d60-0991-4b2a-8bd0-ad0fabe57fe1","Type":"ContainerStarted","Data":"0fb838a709d7e6c9475016ffb36e1d5f914324882d8ef6f911456c4b56fc36b5"} Jul 22 03:34:51.065778 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.065724 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-h2p8d\" (UID: \"4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" Jul 22 03:34:51.067209 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.066754 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" event={"ID":"04756259-81e3-448e-9544-8c71ab94d644","Type":"ContainerStarted","Data":"872c8a22d9473f5bf25f1b4d276f0279b46deca025c06ca0018300520fb7078c"} Jul 22 03:34:51.067209 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.066939 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:51.068224 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.068187 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" Jul 22 03:34:51.068620 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.068602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8-tls-certificates\") pod \"prometheus-operator-admission-webhook-85574b5f49-h2p8d\" (UID: \"4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" Jul 22 03:34:51.069141 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.068827 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" event={"ID":"79981694-7262-47fa-9b35-f1b8bb1f0410","Type":"ContainerStarted","Data":"389828e688d5360cae259d871339551ee55d777c676114ca4d8446184740185b"} Jul 22 03:34:51.070594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.070571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" event={"ID":"e5411e49-cf4d-4c68-96c0-b0d301c58695","Type":"ContainerStarted","Data":"ddf842a367aa9f869acfddc102970f74b2ef469fcf8039cdebb856f554bafd23"} Jul 22 03:34:51.072186 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.072166 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-k9vqm" event={"ID":"200a8fe2-eab4-4e9f-924f-03db74707208","Type":"ContainerStarted","Data":"fc0123bec7a83fbad399720343f996dc32e974cf25209a99c2a7ff62b06b7a39"} Jul 22 03:34:51.074033 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.074014 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-dxbf7" event={"ID":"79027f34-2de1-4e27-9422-43081303a8d8","Type":"ContainerStarted","Data":"76513a410c4ba6340aa5e3056d7702391af8ff4a8c34fc6a96148e871265ec92"} Jul 22 03:34:51.075617 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.075599 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m79rh" event={"ID":"41c2a5d1-2687-40c4-b1dc-869e2f36ee58","Type":"ContainerStarted","Data":"56720b6a246ad55ea6b0e3c2d8b9980dccf13184aedd5e0bc1b4a1f2175b95b3"} Jul 22 03:34:51.077143 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.077123 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx9rw" event={"ID":"670451d7-67ac-46f2-a8ea-af8b553b7602","Type":"ContainerStarted","Data":"3293a1113b132830cb501ccfa63ac5dbc4cbb538ac1a573f3bd2ecc630d72ef5"} Jul 22 03:34:51.094504 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.086800 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-74d568876b-glsjh" podStartSLOduration=2.113296307 podStartE2EDuration="11.086785833s" totalImagesPullingTime="8.973489526s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:51.085991151 +0000 UTC m=+77.879479643" watchObservedRunningTime="2026-07-22 03:34:51.086785833 +0000 UTC m=+77.880274325" Jul 22 03:34:51.123434 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.123381 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-7bc7489cb5-5phsl" podStartSLOduration=2.031829001 podStartE2EDuration="11.123360757s" totalImagesPullingTime="9.091531756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:51.122569647 +0000 UTC m=+77.916058138" watchObservedRunningTime="2026-07-22 03:34:51.123360757 +0000 UTC m=+77.916849249" Jul 22 03:34:51.123539 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.123483 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-7b7bd688cd-klt5h" podStartSLOduration=2.180740391 podStartE2EDuration="11.123475214s" totalImagesPullingTime="8.942734823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:51.105449641 +0000 UTC m=+77.898938130" watchObservedRunningTime="2026-07-22 03:34:51.123475214 +0000 UTC m=+77.916963707" Jul 22 03:34:51.186921 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.186866 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-k9vqm" podStartSLOduration=68.202907058 podStartE2EDuration="1m17.186845956s" totalImagesPullingTime="8.983938898s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:51.186628225 +0000 UTC m=+77.980116715" watchObservedRunningTime="2026-07-22 03:34:51.186845956 +0000 UTC m=+77.980334447" Jul 22 03:34:51.188001 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.187963 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-dxbf7" podStartSLOduration=17.071806267 podStartE2EDuration="34.187947402s" totalImagesPullingTime="17.116141135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:34:17 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:51.16396091 +0000 UTC m=+77.957449401" watchObservedRunningTime="2026-07-22 03:34:51.187947402 +0000 UTC m=+77.981435895" Jul 22 03:34:51.251879 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.251847 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" Jul 22 03:34:51.372210 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:51.372181 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d"] Jul 22 03:34:51.374689 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:34:51.374664 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ab5aa8d_20e1_48b5_8fa3_53b2f3ebecf8.slice/crio-39bfbc02907dddfef96e3d0b09b4116ed5696159fdd614ae77ec9ede57143c3f WatchSource:0}: Error finding container 39bfbc02907dddfef96e3d0b09b4116ed5696159fdd614ae77ec9ede57143c3f: Status 404 returned error can't find the container with id 39bfbc02907dddfef96e3d0b09b4116ed5696159fdd614ae77ec9ede57143c3f Jul 22 03:34:52.084471 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:52.084434 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-m79rh" event={"ID":"41c2a5d1-2687-40c4-b1dc-869e2f36ee58","Type":"ContainerStarted","Data":"0ffd37b738dcd9df642d6d35adc9ac59907b720f7a7fe7d57590ced2c36bb125"} Jul 22 03:34:52.086372 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:52.086292 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-xx9rw" event={"ID":"670451d7-67ac-46f2-a8ea-af8b553b7602","Type":"ContainerStarted","Data":"4c7580942ab4c98c95976cde2eeda4ef3fe74e3d0cc824afc1ae210b8bded417"} Jul 22 03:34:52.086511 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:52.086424 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-xx9rw" Jul 22 03:34:52.087723 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:52.087628 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" event={"ID":"4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8","Type":"ContainerStarted","Data":"39bfbc02907dddfef96e3d0b09b4116ed5696159fdd614ae77ec9ede57143c3f"} Jul 22 03:34:52.102243 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:52.102196 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-m79rh" podStartSLOduration=70.239656619 podStartE2EDuration="1m18.102181612s" totalImagesPullingTime="7.862524993s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:33:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:52.100385795 +0000 UTC m=+78.893874285" watchObservedRunningTime="2026-07-22 03:34:52.102181612 +0000 UTC m=+78.895670104" Jul 22 03:34:52.119125 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:52.119080 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-xx9rw" podStartSLOduration=35.175324808 podStartE2EDuration="44.119064228s" totalImagesPullingTime="8.94373942s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:52.118298376 +0000 UTC m=+78.911786881" watchObservedRunningTime="2026-07-22 03:34:52.119064228 +0000 UTC m=+78.912552718" Jul 22 03:34:53.093036 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:53.092999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" event={"ID":"4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8","Type":"ContainerStarted","Data":"1de2bc25338d132f78e6850d6f2dcfe56992521ebd3182a9f8d985c8d9e712c8"} Jul 22 03:34:53.093717 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:53.093701 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" Jul 22 03:34:53.098661 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:53.098638 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" Jul 22 03:34:53.111000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:53.110962 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-85574b5f49-h2p8d" podStartSLOduration=1.9754485640000001 podStartE2EDuration="3.110942831s" totalImagesPullingTime="1.135494267s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:53.109504693 +0000 UTC m=+79.902993184" watchObservedRunningTime="2026-07-22 03:34:53.110942831 +0000 UTC m=+79.904431322" Jul 22 03:34:54.098013 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:54.097890 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f48b846d-8n5kd" event={"ID":"82935d60-0991-4b2a-8bd0-ad0fabe57fe1","Type":"ContainerStarted","Data":"4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af"} Jul 22 03:34:54.119578 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:54.119537 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64f48b846d-8n5kd" podStartSLOduration=7.075201733 podStartE2EDuration="10.119522853s" totalImagesPullingTime="3.04432112s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:34:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:34:54.117510021 +0000 UTC m=+80.910998514" watchObservedRunningTime="2026-07-22 03:34:54.119522853 +0000 UTC m=+80.913011344" Jul 22 03:34:54.831740 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:54.831712 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:54.831876 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:54.831768 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:54.835949 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:54.835929 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:34:55.104885 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:34:55.104826 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:35:02.095033 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:02.095006 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-xx9rw" Jul 22 03:35:08.248113 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.248073 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j"] Jul 22 03:35:08.251632 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.251613 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.255129 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.255106 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 03:35:08.255265 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.255247 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 22 03:35:08.255416 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.255138 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 22 03:35:08.255514 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.255122 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-tlxf6\"" Jul 22 03:35:08.255938 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.255920 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 03:35:08.256574 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.256559 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 03:35:08.263071 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.263049 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j"] Jul 22 03:35:08.275544 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.275525 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fzsxx"] Jul 22 03:35:08.277557 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.277533 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.280472 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.280453 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 03:35:08.280642 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.280627 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hd5r7\"" Jul 22 03:35:08.281280 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.281253 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 03:35:08.281405 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.281329 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 03:35:08.373097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373071 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/784e29f1-92ee-473d-9b46-38564caffd18-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.373196 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373102 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl4wt\" (UniqueName: \"kubernetes.io/projected/c3e77fc6-be39-4cd3-8f43-515b89545689-kube-api-access-gl4wt\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373196 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373119 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-wtmp\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373196 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373134 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-sys\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373234 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373303 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-tls\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373458 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373356 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3e77fc6-be39-4cd3-8f43-515b89545689-metrics-client-ca\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373458 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373390 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.373458 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9vgt\" (UniqueName: \"kubernetes.io/projected/784e29f1-92ee-473d-9b46-38564caffd18-kube-api-access-g9vgt\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.373594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373456 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-root\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.373594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.373594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.373535 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-textfile\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474193 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474170 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/784e29f1-92ee-473d-9b46-38564caffd18-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.474283 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474205 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gl4wt\" (UniqueName: \"kubernetes.io/projected/c3e77fc6-be39-4cd3-8f43-515b89545689-kube-api-access-gl4wt\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474382 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474295 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-wtmp\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474382 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474358 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-sys\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474488 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474419 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474488 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-tls\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474576 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474491 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3e77fc6-be39-4cd3-8f43-515b89545689-metrics-client-ca\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474576 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474503 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-sys\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474576 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474537 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.474576 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474572 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g9vgt\" (UniqueName: \"kubernetes.io/projected/784e29f1-92ee-473d-9b46-38564caffd18-kube-api-access-g9vgt\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.474763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474617 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474628 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-wtmp\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474645 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-root\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474690 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.474763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-textfile\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.474985 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.474927 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/784e29f1-92ee-473d-9b46-38564caffd18-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.475211 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.475188 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c3e77fc6-be39-4cd3-8f43-515b89545689-metrics-client-ca\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.475354 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:35:08.475317 2564 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 22 03:35:08.475426 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.475368 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.475426 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:35:08.475403 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-tls podName:c3e77fc6-be39-4cd3-8f43-515b89545689 nodeName:}" failed. No retries permitted until 2026-07-22 03:35:08.975384754 +0000 UTC m=+95.768873237 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-tls") pod "node-exporter-fzsxx" (UID: "c3e77fc6-be39-4cd3-8f43-515b89545689") : secret "node-exporter-tls" not found Jul 22 03:35:08.475529 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.475432 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-textfile\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.475529 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.475438 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c3e77fc6-be39-4cd3-8f43-515b89545689-root\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.475718 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:35:08.475704 2564 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 22 03:35:08.475796 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:35:08.475750 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-tls podName:784e29f1-92ee-473d-9b46-38564caffd18 nodeName:}" failed. No retries permitted until 2026-07-22 03:35:08.975736987 +0000 UTC m=+95.769225469 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-rqs7j" (UID: "784e29f1-92ee-473d-9b46-38564caffd18") : secret "openshift-state-metrics-tls" not found Jul 22 03:35:08.477887 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.477867 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.478042 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.478023 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.484784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.484738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9vgt\" (UniqueName: \"kubernetes.io/projected/784e29f1-92ee-473d-9b46-38564caffd18-kube-api-access-g9vgt\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.485607 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.485566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl4wt\" (UniqueName: \"kubernetes.io/projected/c3e77fc6-be39-4cd3-8f43-515b89545689-kube-api-access-gl4wt\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.978005 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.977981 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:08.978119 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.978029 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-tls\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.980141 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.980122 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c3e77fc6-be39-4cd3-8f43-515b89545689-node-exporter-tls\") pod \"node-exporter-fzsxx\" (UID: \"c3e77fc6-be39-4cd3-8f43-515b89545689\") " pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:08.980384 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:08.980366 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/784e29f1-92ee-473d-9b46-38564caffd18-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-rqs7j\" (UID: \"784e29f1-92ee-473d-9b46-38564caffd18\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:09.164196 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.164175 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" Jul 22 03:35:09.186877 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.186857 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fzsxx" Jul 22 03:35:09.217368 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:35:09.217318 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3e77fc6_be39_4cd3_8f43_515b89545689.slice/crio-4af1398c9aaf0eb426ae6deed089f3e1938fcde0e4d9e8a65dcbb0c0c006396c WatchSource:0}: Error finding container 4af1398c9aaf0eb426ae6deed089f3e1938fcde0e4d9e8a65dcbb0c0c006396c: Status 404 returned error can't find the container with id 4af1398c9aaf0eb426ae6deed089f3e1938fcde0e4d9e8a65dcbb0c0c006396c Jul 22 03:35:09.331999 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.331969 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j"] Jul 22 03:35:09.335772 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:35:09.335673 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod784e29f1_92ee_473d_9b46_38564caffd18.slice/crio-4af7cc8f7163f3706820a1ce4279c67a1eba14cac8c17c8b94ea6248464708e5 WatchSource:0}: Error finding container 4af7cc8f7163f3706820a1ce4279c67a1eba14cac8c17c8b94ea6248464708e5: Status 404 returned error can't find the container with id 4af7cc8f7163f3706820a1ce4279c67a1eba14cac8c17c8b94ea6248464708e5 Jul 22 03:35:09.414021 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.413997 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 03:35:09.440841 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.440816 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 03:35:09.440984 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.440963 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.444145 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.444121 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 22 03:35:09.444415 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.444390 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 22 03:35:09.444415 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.444410 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 22 03:35:09.444916 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.444746 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 22 03:35:09.445017 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.444921 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 22 03:35:09.445080 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.445036 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 22 03:35:09.445704 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.445678 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 22 03:35:09.445783 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.445749 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 22 03:35:09.445869 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.445784 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-jlwv7\"" Jul 22 03:35:09.448183 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.448118 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 22 03:35:09.582843 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.582819 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c839d350-f629-4775-9b9f-b43945f3eddd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.582959 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.582863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.582959 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.582930 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583068 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.582959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839d350-f629-4775-9b9f-b43945f3eddd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583068 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583004 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c839d350-f629-4775-9b9f-b43945f3eddd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583068 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583031 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583068 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k9424\" (UniqueName: \"kubernetes.io/projected/c839d350-f629-4775-9b9f-b43945f3eddd-kube-api-access-k9424\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583079 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c839d350-f629-4775-9b9f-b43945f3eddd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583111 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583149 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583192 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c839d350-f629-4775-9b9f-b43945f3eddd-config-out\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583225 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-config-volume\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.583255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.583248 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-web-config\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684226 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684192 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c839d350-f629-4775-9b9f-b43945f3eddd-config-out\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684370 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-config-volume\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684370 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684253 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-web-config\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684370 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684279 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c839d350-f629-4775-9b9f-b43945f3eddd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684370 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684307 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684554 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684374 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684554 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684399 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839d350-f629-4775-9b9f-b43945f3eddd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684554 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684424 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c839d350-f629-4775-9b9f-b43945f3eddd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684554 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684935 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684613 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k9424\" (UniqueName: \"kubernetes.io/projected/c839d350-f629-4775-9b9f-b43945f3eddd-kube-api-access-k9424\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684935 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c839d350-f629-4775-9b9f-b43945f3eddd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684935 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684703 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.684935 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.684763 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.685265 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.685201 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c839d350-f629-4775-9b9f-b43945f3eddd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.686633 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.686300 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c839d350-f629-4775-9b9f-b43945f3eddd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.686633 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.686441 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c839d350-f629-4775-9b9f-b43945f3eddd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.687735 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.687668 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-config-volume\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.688084 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.688036 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c839d350-f629-4775-9b9f-b43945f3eddd-config-out\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.688396 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.688316 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.688701 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.688666 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.689754 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.689613 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-web-config\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.689865 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.689764 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.690413 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.690116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c839d350-f629-4775-9b9f-b43945f3eddd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.690413 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.690378 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.690850 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.690648 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c839d350-f629-4775-9b9f-b43945f3eddd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.695092 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.695070 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k9424\" (UniqueName: \"kubernetes.io/projected/c839d350-f629-4775-9b9f-b43945f3eddd-kube-api-access-k9424\") pod \"alertmanager-main-0\" (UID: \"c839d350-f629-4775-9b9f-b43945f3eddd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.752044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.752022 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 22 03:35:09.916114 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:09.916093 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 22 03:35:09.918247 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:35:09.918214 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc839d350_f629_4775_9b9f_b43945f3eddd.slice/crio-97ce1e5a254fb09ec9e6f1fb46d0b0a62300f17ad2934643b3cf610064502ead WatchSource:0}: Error finding container 97ce1e5a254fb09ec9e6f1fb46d0b0a62300f17ad2934643b3cf610064502ead: Status 404 returned error can't find the container with id 97ce1e5a254fb09ec9e6f1fb46d0b0a62300f17ad2934643b3cf610064502ead Jul 22 03:35:10.146852 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:10.146772 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzsxx" event={"ID":"c3e77fc6-be39-4cd3-8f43-515b89545689","Type":"ContainerStarted","Data":"4af1398c9aaf0eb426ae6deed089f3e1938fcde0e4d9e8a65dcbb0c0c006396c"} Jul 22 03:35:10.147946 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:10.147920 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerStarted","Data":"97ce1e5a254fb09ec9e6f1fb46d0b0a62300f17ad2934643b3cf610064502ead"} Jul 22 03:35:10.149430 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:10.149409 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" event={"ID":"784e29f1-92ee-473d-9b46-38564caffd18","Type":"ContainerStarted","Data":"2392637e416bc8653855b9ae752a049c7323fe58ab4310aa898be7c3c49dc06d"} Jul 22 03:35:10.149589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:10.149447 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" event={"ID":"784e29f1-92ee-473d-9b46-38564caffd18","Type":"ContainerStarted","Data":"dbff93e8c97dde7ffc3a04e01170f9390f2efa9e598177179ab634260bed3e3b"} Jul 22 03:35:10.149589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:10.149464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" event={"ID":"784e29f1-92ee-473d-9b46-38564caffd18","Type":"ContainerStarted","Data":"4af7cc8f7163f3706820a1ce4279c67a1eba14cac8c17c8b94ea6248464708e5"} Jul 22 03:35:11.155236 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:11.155203 2564 generic.go:356] "Generic (PLEG): container finished" podID="c3e77fc6-be39-4cd3-8f43-515b89545689" containerID="423ae2a42da5bdadd2d7e6f03ba99f2f27f74fdf6845cc9c7b418b233a61873f" exitCode=0 Jul 22 03:35:11.155690 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:11.155249 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzsxx" event={"ID":"c3e77fc6-be39-4cd3-8f43-515b89545689","Type":"ContainerDied","Data":"423ae2a42da5bdadd2d7e6f03ba99f2f27f74fdf6845cc9c7b418b233a61873f"} Jul 22 03:35:12.161659 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.161623 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzsxx" event={"ID":"c3e77fc6-be39-4cd3-8f43-515b89545689","Type":"ContainerStarted","Data":"3755017d282712e7ec1fc321aec64699201e88b2c86ebbdf62583c303de12181"} Jul 22 03:35:12.161984 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.161692 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzsxx" event={"ID":"c3e77fc6-be39-4cd3-8f43-515b89545689","Type":"ContainerStarted","Data":"f427d4681d19f59b52f960685d791fc9f439b31d829e5b2325b63b1fe3b809dc"} Jul 22 03:35:12.163195 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.163168 2564 generic.go:356] "Generic (PLEG): container finished" podID="c839d350-f629-4775-9b9f-b43945f3eddd" containerID="a3bf1e737ec79eb5946f73029241fb09be792dbe31a3536607b1886ae06b7d59" exitCode=0 Jul 22 03:35:12.163304 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.163244 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerDied","Data":"a3bf1e737ec79eb5946f73029241fb09be792dbe31a3536607b1886ae06b7d59"} Jul 22 03:35:12.165741 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.165707 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" event={"ID":"784e29f1-92ee-473d-9b46-38564caffd18","Type":"ContainerStarted","Data":"0d5061f46b03a398ce3bbe6e90bb6a95832809465cd7ae06c540195008179d56"} Jul 22 03:35:12.184551 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.184504 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fzsxx" podStartSLOduration=2.793652358 podStartE2EDuration="4.18449243s" totalImagesPullingTime="1.284532307s" totalInitContainerRuntime="106.307765ms" isStatefulPod=true podCreationTimestamp="2026-07-22 03:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:12.182891563 +0000 UTC m=+98.976380056" watchObservedRunningTime="2026-07-22 03:35:12.18449243 +0000 UTC m=+98.977980943" Jul 22 03:35:12.200759 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.200722 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-rqs7j" podStartSLOduration=1.91471491 podStartE2EDuration="4.200710583s" totalImagesPullingTime="2.285995673s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:12.199657548 +0000 UTC m=+98.993146038" watchObservedRunningTime="2026-07-22 03:35:12.200710583 +0000 UTC m=+98.994199073" Jul 22 03:35:12.624144 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:12.624122 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f48b846d-8n5kd"] Jul 22 03:35:13.030638 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.030608 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv"] Jul 22 03:35:13.055950 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.055923 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv"] Jul 22 03:35:13.056104 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.056024 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-gkdws" Jul 22 03:35:13.056104 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.056066 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.059100 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.059080 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 22 03:35:13.059606 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.059587 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-p85z4\"" Jul 22 03:35:13.212491 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.212460 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a01b5c89-22e1-47f2-a625-de9c3b7f8681-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-lxhlv\" (UID: \"a01b5c89-22e1-47f2-a625-de9c3b7f8681\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.212870 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.212554 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7k6hz\" (UniqueName: \"kubernetes.io/projected/a01b5c89-22e1-47f2-a625-de9c3b7f8681-kube-api-access-7k6hz\") pod \"monitoring-plugin-7b7b7bc8bf-lxhlv\" (UID: \"a01b5c89-22e1-47f2-a625-de9c3b7f8681\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.313260 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.313193 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7k6hz\" (UniqueName: \"kubernetes.io/projected/a01b5c89-22e1-47f2-a625-de9c3b7f8681-kube-api-access-7k6hz\") pod \"monitoring-plugin-7b7b7bc8bf-lxhlv\" (UID: \"a01b5c89-22e1-47f2-a625-de9c3b7f8681\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.313563 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.313528 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a01b5c89-22e1-47f2-a625-de9c3b7f8681-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-lxhlv\" (UID: \"a01b5c89-22e1-47f2-a625-de9c3b7f8681\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.313661 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:35:13.313601 2564 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 22 03:35:13.313712 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:35:13.313664 2564 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a01b5c89-22e1-47f2-a625-de9c3b7f8681-monitoring-plugin-cert podName:a01b5c89-22e1-47f2-a625-de9c3b7f8681 nodeName:}" failed. No retries permitted until 2026-07-22 03:35:13.813642325 +0000 UTC m=+100.607130810 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/a01b5c89-22e1-47f2-a625-de9c3b7f8681-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-lxhlv" (UID: "a01b5c89-22e1-47f2-a625-de9c3b7f8681") : secret "monitoring-plugin-cert" not found Jul 22 03:35:13.323046 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.323025 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7k6hz\" (UniqueName: \"kubernetes.io/projected/a01b5c89-22e1-47f2-a625-de9c3b7f8681-kube-api-access-7k6hz\") pod \"monitoring-plugin-7b7b7bc8bf-lxhlv\" (UID: \"a01b5c89-22e1-47f2-a625-de9c3b7f8681\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.817313 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.817281 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a01b5c89-22e1-47f2-a625-de9c3b7f8681-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-lxhlv\" (UID: \"a01b5c89-22e1-47f2-a625-de9c3b7f8681\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.820253 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.820227 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/a01b5c89-22e1-47f2-a625-de9c3b7f8681-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-lxhlv\" (UID: \"a01b5c89-22e1-47f2-a625-de9c3b7f8681\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:13.967651 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:13.967619 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:14.094997 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.093594 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8b599978-jj8mr"] Jul 22 03:35:14.103881 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.103851 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.107547 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.107233 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b599978-jj8mr"] Jul 22 03:35:14.113626 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.113545 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 03:35:14.124254 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.124232 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv"] Jul 22 03:35:14.129369 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:35:14.129326 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda01b5c89_22e1_47f2_a625_de9c3b7f8681.slice/crio-b9e4ba204d059bf4d7a34d008962632b2ed562a4e8e08fbf59e3af0d394417ab WatchSource:0}: Error finding container b9e4ba204d059bf4d7a34d008962632b2ed562a4e8e08fbf59e3af0d394417ab: Status 404 returned error can't find the container with id b9e4ba204d059bf4d7a34d008962632b2ed562a4e8e08fbf59e3af0d394417ab Jul 22 03:35:14.175153 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.175120 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" event={"ID":"a01b5c89-22e1-47f2-a625-de9c3b7f8681","Type":"ContainerStarted","Data":"b9e4ba204d059bf4d7a34d008962632b2ed562a4e8e08fbf59e3af0d394417ab"} Jul 22 03:35:14.177501 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.177482 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerStarted","Data":"ca5cda579680fecc283651bc3575e14917a44f32c7d39f51fbdbe3dc3ee02ccc"} Jul 22 03:35:14.177607 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.177545 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerStarted","Data":"9d539473c3679aa08a84c620ea8324d80be5dff2e512b238114e4d916a002999"} Jul 22 03:35:14.221783 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.221763 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-oauth-serving-cert\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.222134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.221804 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-oauth-config\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.222134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.221863 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-config\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.222134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.221916 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-serving-cert\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.222134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.221933 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-service-ca\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.222134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.221949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-trusted-ca-bundle\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.222134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.222003 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xpnb6\" (UniqueName: \"kubernetes.io/projected/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-kube-api-access-xpnb6\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.322427 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.322371 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-oauth-config\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.322427 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.322407 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-config\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.322587 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.322434 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-serving-cert\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.322587 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.322448 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-service-ca\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.322587 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.322464 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-trusted-ca-bundle\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.322587 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.322486 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xpnb6\" (UniqueName: \"kubernetes.io/projected/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-kube-api-access-xpnb6\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.322587 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.322536 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-oauth-serving-cert\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.323237 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.323184 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-oauth-serving-cert\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.323378 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.323264 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-service-ca\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.323445 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.323388 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-trusted-ca-bundle\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.323445 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.323391 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-config\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.324836 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.324819 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-serving-cert\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.324903 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.324874 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-oauth-config\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.330451 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.330429 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xpnb6\" (UniqueName: \"kubernetes.io/projected/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-kube-api-access-xpnb6\") pod \"console-8b599978-jj8mr\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.432576 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.432558 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:14.556889 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:14.556864 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8b599978-jj8mr"] Jul 22 03:35:14.558629 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:35:14.558606 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode845ea4a_9a7b_45f0_a08e_0bb8805a9365.slice/crio-8abe772edf106d2ce3222cb60c59ba358e80a540772714d847adb54423cd90f4 WatchSource:0}: Error finding container 8abe772edf106d2ce3222cb60c59ba358e80a540772714d847adb54423cd90f4: Status 404 returned error can't find the container with id 8abe772edf106d2ce3222cb60c59ba358e80a540772714d847adb54423cd90f4 Jul 22 03:35:15.186418 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:15.186378 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerStarted","Data":"1dc68473715ce093d4d7c314b7d6195c01e36361bbc149e2d217224d2b464dfa"} Jul 22 03:35:15.186592 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:15.186458 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerStarted","Data":"a2f8de87a0a0bf7d34485c785500bff22fd77ad396035e75b10943611a899d61"} Jul 22 03:35:15.186592 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:15.186499 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerStarted","Data":"ae6725489c4cd85fb63139b8aa8efbaaa749a5f9fb802dadb61a508008b798de"} Jul 22 03:35:15.187909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:15.187877 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b599978-jj8mr" event={"ID":"e845ea4a-9a7b-45f0-a08e-0bb8805a9365","Type":"ContainerStarted","Data":"67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639"} Jul 22 03:35:15.187909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:15.187922 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b599978-jj8mr" event={"ID":"e845ea4a-9a7b-45f0-a08e-0bb8805a9365","Type":"ContainerStarted","Data":"8abe772edf106d2ce3222cb60c59ba358e80a540772714d847adb54423cd90f4"} Jul 22 03:35:15.208401 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:15.208329 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8b599978-jj8mr" podStartSLOduration=1.208309426 podStartE2EDuration="1.208309426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:15.20673079 +0000 UTC m=+102.000219281" watchObservedRunningTime="2026-07-22 03:35:15.208309426 +0000 UTC m=+102.001797919" Jul 22 03:35:16.196494 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:16.196467 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" event={"ID":"a01b5c89-22e1-47f2-a625-de9c3b7f8681","Type":"ContainerStarted","Data":"cfef2a6f27f64a7cbb31f586f5a1a35b1e9baf77030e10b305db6df9207bedec"} Jul 22 03:35:16.196889 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:16.196662 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:16.199978 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:16.199953 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c839d350-f629-4775-9b9f-b43945f3eddd","Type":"ContainerStarted","Data":"ba771539dde77fe388bcb43f3b428249f2129ea1fd5ccd472f47138c6845b295"} Jul 22 03:35:16.201779 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:16.201763 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" Jul 22 03:35:16.213924 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:16.213886 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-lxhlv" podStartSLOduration=1.757643534 podStartE2EDuration="3.213875726s" totalImagesPullingTime="1.456232192s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:16.213506887 +0000 UTC m=+103.006995380" watchObservedRunningTime="2026-07-22 03:35:16.213875726 +0000 UTC m=+103.007364216" Jul 22 03:35:16.242555 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:16.242512 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.174045969 podStartE2EDuration="7.242497261s" totalImagesPullingTime="5.061202876s" totalInitContainerRuntime="7.248416ms" isStatefulPod=false podCreationTimestamp="2026-07-22 03:35:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:35:16.240656025 +0000 UTC m=+103.034144537" watchObservedRunningTime="2026-07-22 03:35:16.242497261 +0000 UTC m=+103.035985753" Jul 22 03:35:24.433356 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:24.433300 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:24.433762 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:24.433398 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:24.437906 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:24.437886 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:25.235114 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:25.235086 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:35:37.644842 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:37.644782 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64f48b846d-8n5kd" podUID="82935d60-0991-4b2a-8bd0-ad0fabe57fe1" containerName="console" containerID="cri-o://4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af" gracePeriod=15 Jul 22 03:35:37.873237 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:37.873212 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:35:37.900820 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:37.900763 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-service-ca\") pod \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " Jul 22 03:35:37.901100 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:37.901077 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-service-ca" (OuterVolumeSpecName: "service-ca") pod "82935d60-0991-4b2a-8bd0-ad0fabe57fe1" (UID: "82935d60-0991-4b2a-8bd0-ad0fabe57fe1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:35:38.001202 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001183 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-oauth-config\") pod \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " Jul 22 03:35:38.001306 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001213 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-serving-cert\") pod \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " Jul 22 03:35:38.001306 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001237 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-oauth-serving-cert\") pod \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " Jul 22 03:35:38.001306 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001265 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vpw94\" (UniqueName: \"kubernetes.io/projected/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-kube-api-access-vpw94\") pod \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " Jul 22 03:35:38.001306 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001292 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-config\") pod \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\" (UID: \"82935d60-0991-4b2a-8bd0-ad0fabe57fe1\") " Jul 22 03:35:38.001531 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001511 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:35:38.001616 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001581 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "82935d60-0991-4b2a-8bd0-ad0fabe57fe1" (UID: "82935d60-0991-4b2a-8bd0-ad0fabe57fe1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:35:38.001735 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.001652 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-config" (OuterVolumeSpecName: "console-config") pod "82935d60-0991-4b2a-8bd0-ad0fabe57fe1" (UID: "82935d60-0991-4b2a-8bd0-ad0fabe57fe1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:35:38.003584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.003557 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "82935d60-0991-4b2a-8bd0-ad0fabe57fe1" (UID: "82935d60-0991-4b2a-8bd0-ad0fabe57fe1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:35:38.003584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.003560 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-kube-api-access-vpw94" (OuterVolumeSpecName: "kube-api-access-vpw94") pod "82935d60-0991-4b2a-8bd0-ad0fabe57fe1" (UID: "82935d60-0991-4b2a-8bd0-ad0fabe57fe1"). InnerVolumeSpecName "kube-api-access-vpw94". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:35:38.003692 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.003628 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "82935d60-0991-4b2a-8bd0-ad0fabe57fe1" (UID: "82935d60-0991-4b2a-8bd0-ad0fabe57fe1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:35:38.101773 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.101757 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-oauth-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:35:38.101773 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.101774 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:35:38.101884 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.101783 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-oauth-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:35:38.101884 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.101791 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vpw94\" (UniqueName: \"kubernetes.io/projected/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-kube-api-access-vpw94\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:35:38.101884 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.101799 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/82935d60-0991-4b2a-8bd0-ad0fabe57fe1-console-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:35:38.281926 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.281907 2564 generic.go:356] "Generic (PLEG): container finished" podID="82935d60-0991-4b2a-8bd0-ad0fabe57fe1" containerID="4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af" exitCode=0 Jul 22 03:35:38.282002 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.281951 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f48b846d-8n5kd" Jul 22 03:35:38.282002 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.281989 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f48b846d-8n5kd" event={"ID":"82935d60-0991-4b2a-8bd0-ad0fabe57fe1","Type":"ContainerDied","Data":"4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af"} Jul 22 03:35:38.282072 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.282041 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f48b846d-8n5kd" event={"ID":"82935d60-0991-4b2a-8bd0-ad0fabe57fe1","Type":"ContainerDied","Data":"0fb838a709d7e6c9475016ffb36e1d5f914324882d8ef6f911456c4b56fc36b5"} Jul 22 03:35:38.282072 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.282065 2564 scope.go:117] "RemoveContainer" containerID="4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af" Jul 22 03:35:38.290469 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.290450 2564 scope.go:117] "RemoveContainer" containerID="4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af" Jul 22 03:35:38.290727 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:35:38.290706 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af\": container with ID starting with 4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af not found: ID does not exist" containerID="4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af" Jul 22 03:35:38.290791 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.290739 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af"} err="failed to get container status \"4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af\": rpc error: code = NotFound desc = could not find container \"4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af\": container with ID starting with 4dacc0861d61f63f8c1edecca64463c41b189c4a5acc51650a85e641a7ee45af not found: ID does not exist" Jul 22 03:35:38.303576 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.303554 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f48b846d-8n5kd"] Jul 22 03:35:38.307189 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:38.307168 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64f48b846d-8n5kd"] Jul 22 03:35:39.743751 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:35:39.743725 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82935d60-0991-4b2a-8bd0-ad0fabe57fe1" path="/var/lib/kubelet/pods/82935d60-0991-4b2a-8bd0-ad0fabe57fe1/volumes" Jul 22 03:36:32.638998 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.638920 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-76586b478c-9bvln"] Jul 22 03:36:32.639430 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.639251 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82935d60-0991-4b2a-8bd0-ad0fabe57fe1" containerName="console" Jul 22 03:36:32.639430 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.639264 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="82935d60-0991-4b2a-8bd0-ad0fabe57fe1" containerName="console" Jul 22 03:36:32.639430 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.639312 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="82935d60-0991-4b2a-8bd0-ad0fabe57fe1" containerName="console" Jul 22 03:36:32.641324 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.641308 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.646446 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.646421 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 22 03:36:32.646732 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.646702 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 22 03:36:32.646841 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.646762 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 22 03:36:32.646841 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.646777 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jtlnh\"" Jul 22 03:36:32.646933 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.646848 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 22 03:36:32.647988 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.647970 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 22 03:36:32.653212 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.653194 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 22 03:36:32.658701 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.658682 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76586b478c-9bvln"] Jul 22 03:36:32.793988 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.793964 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-metrics-client-ca\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.794083 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.794001 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.794083 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.794029 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-federate-client-tls\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.794152 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.794077 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-secret-telemeter-client\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.794152 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.794138 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.794214 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.794157 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-telemeter-client-tls\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.794214 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.794196 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-serving-certs-ca-bundle\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.794276 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.794213 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89gxc\" (UniqueName: \"kubernetes.io/projected/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-kube-api-access-89gxc\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.894936 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.894862 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-metrics-client-ca\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.894936 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.894919 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.895119 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.894968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-federate-client-tls\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.895119 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.895000 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-secret-telemeter-client\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.895119 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.895050 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.895119 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.895074 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-telemeter-client-tls\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.895303 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.895129 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-serving-certs-ca-bundle\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.895303 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.895162 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89gxc\" (UniqueName: \"kubernetes.io/projected/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-kube-api-access-89gxc\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.895874 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.895813 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-metrics-client-ca\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.896151 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.896116 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-serving-certs-ca-bundle\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.896641 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.896617 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-telemeter-trusted-ca-bundle\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.898004 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.897985 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-secret-telemeter-client\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.898220 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.898198 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-federate-client-tls\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.898261 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.898208 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.898667 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.898649 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-telemeter-client-tls\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.904543 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.904522 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89gxc\" (UniqueName: \"kubernetes.io/projected/8cb7f48a-f9bd-4ce9-b146-0e7058390e92-kube-api-access-89gxc\") pod \"telemeter-client-76586b478c-9bvln\" (UID: \"8cb7f48a-f9bd-4ce9-b146-0e7058390e92\") " pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:32.951044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:32.951011 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" Jul 22 03:36:33.085062 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:33.085037 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-76586b478c-9bvln"] Jul 22 03:36:33.086361 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:36:33.086306 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cb7f48a_f9bd_4ce9_b146_0e7058390e92.slice/crio-6c2779d88d37bea6f00cf2852dd8a0d0982b652d1fa0182b4a90fc9d0759831f WatchSource:0}: Error finding container 6c2779d88d37bea6f00cf2852dd8a0d0982b652d1fa0182b4a90fc9d0759831f: Status 404 returned error can't find the container with id 6c2779d88d37bea6f00cf2852dd8a0d0982b652d1fa0182b4a90fc9d0759831f Jul 22 03:36:33.487705 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:33.487674 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" event={"ID":"8cb7f48a-f9bd-4ce9-b146-0e7058390e92","Type":"ContainerStarted","Data":"6c2779d88d37bea6f00cf2852dd8a0d0982b652d1fa0182b4a90fc9d0759831f"} Jul 22 03:36:35.496697 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:35.496656 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" event={"ID":"8cb7f48a-f9bd-4ce9-b146-0e7058390e92","Type":"ContainerStarted","Data":"6bf501c8b5319e0a727a347ad3edccdf4852edadd7835949fd0e5f1a7e4929d9"} Jul 22 03:36:35.497091 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:35.496725 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" event={"ID":"8cb7f48a-f9bd-4ce9-b146-0e7058390e92","Type":"ContainerStarted","Data":"574d39f0edc96e01335cd396a90caf6d72b17e2ce1a2b456abc15a1b9d8688cb"} Jul 22 03:36:35.497091 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:35.496755 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" event={"ID":"8cb7f48a-f9bd-4ce9-b146-0e7058390e92","Type":"ContainerStarted","Data":"369080d2f191b2b311da421b2725eb98ab419ad247786694801c3dd34533db3c"} Jul 22 03:36:35.522195 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:35.522146 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-76586b478c-9bvln" podStartSLOduration=1.581348968 podStartE2EDuration="3.522132623s" totalImagesPullingTime="1.940783655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:36:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:36:35.52088091 +0000 UTC m=+182.314369402" watchObservedRunningTime="2026-07-22 03:36:35.522132623 +0000 UTC m=+182.315621114" Jul 22 03:36:36.313313 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.313282 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-584f576d4c-v4htn"] Jul 22 03:36:36.315492 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.315476 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.331696 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.331661 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-584f576d4c-v4htn"] Jul 22 03:36:36.422749 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.422711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-oauth-serving-cert\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.422749 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.422753 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-trusted-ca-bundle\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.422977 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.422781 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-service-ca\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.422977 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.422867 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfpmn\" (UniqueName: \"kubernetes.io/projected/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-kube-api-access-qfpmn\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.422977 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.422917 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-oauth-config\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.422977 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.422946 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-config\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.422977 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.422971 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-serving-cert\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.524199 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.524157 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfpmn\" (UniqueName: \"kubernetes.io/projected/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-kube-api-access-qfpmn\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.524673 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.524208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-oauth-config\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.524673 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.524244 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-config\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.524673 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.524274 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-serving-cert\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.524673 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.524313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-oauth-serving-cert\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.524673 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.524473 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-trusted-ca-bundle\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.524673 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.524535 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-service-ca\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.525111 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.525079 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-config\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.525211 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.525169 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-oauth-serving-cert\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.525324 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.525307 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-trusted-ca-bundle\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.525392 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.525370 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-service-ca\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.527048 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.527018 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-oauth-config\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.527133 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.527043 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-serving-cert\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.534764 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.534743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfpmn\" (UniqueName: \"kubernetes.io/projected/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-kube-api-access-qfpmn\") pod \"console-584f576d4c-v4htn\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.624683 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.624594 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:36.764688 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:36.764652 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-584f576d4c-v4htn"] Jul 22 03:36:36.768298 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:36:36.768269 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9846fa3b_5928_4ac6_9100_d9633e2c0c9b.slice/crio-02a5ea1f84ca3fc6663110a2964e16d1582ddf9bbd6f2359fb6d5a01a1fa6620 WatchSource:0}: Error finding container 02a5ea1f84ca3fc6663110a2964e16d1582ddf9bbd6f2359fb6d5a01a1fa6620: Status 404 returned error can't find the container with id 02a5ea1f84ca3fc6663110a2964e16d1582ddf9bbd6f2359fb6d5a01a1fa6620 Jul 22 03:36:37.505377 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:37.505329 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584f576d4c-v4htn" event={"ID":"9846fa3b-5928-4ac6-9100-d9633e2c0c9b","Type":"ContainerStarted","Data":"7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9"} Jul 22 03:36:37.505584 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:37.505396 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584f576d4c-v4htn" event={"ID":"9846fa3b-5928-4ac6-9100-d9633e2c0c9b","Type":"ContainerStarted","Data":"02a5ea1f84ca3fc6663110a2964e16d1582ddf9bbd6f2359fb6d5a01a1fa6620"} Jul 22 03:36:37.524956 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:37.524900 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-584f576d4c-v4htn" podStartSLOduration=1.524884309 podStartE2EDuration="1.524884309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:36:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:36:37.523704584 +0000 UTC m=+184.317193088" watchObservedRunningTime="2026-07-22 03:36:37.524884309 +0000 UTC m=+184.318372800" Jul 22 03:36:46.625320 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:46.625288 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:46.625833 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:46.625381 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:46.630118 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:46.630094 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:47.513789 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.513753 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-584f576d4c-v4htn"] Jul 22 03:36:47.546884 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.546854 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64f8b8b4f7-rhzk9"] Jul 22 03:36:47.549164 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.549149 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:36:47.549255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.549240 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.561752 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.561729 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f8b8b4f7-rhzk9"] Jul 22 03:36:47.721003 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.720975 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-oauth-serving-cert\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.721355 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.721008 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-service-ca\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.721355 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.721064 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-config\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.721355 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.721214 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-serving-cert\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.721355 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.721304 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-trusted-ca-bundle\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.721552 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.721356 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2ktw\" (UniqueName: \"kubernetes.io/projected/570bd4fa-1209-4755-a7f4-a60b048ad9a3-kube-api-access-g2ktw\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.721552 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.721389 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-oauth-config\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.822513 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.822463 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-serving-cert\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.822513 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.822499 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-trusted-ca-bundle\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.822649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.822515 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2ktw\" (UniqueName: \"kubernetes.io/projected/570bd4fa-1209-4755-a7f4-a60b048ad9a3-kube-api-access-g2ktw\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.822649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.822532 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-oauth-config\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.822753 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.822663 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-oauth-serving-cert\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.822753 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.822717 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-service-ca\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.822847 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.822764 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-config\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.823442 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.823422 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-config\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.823563 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.823547 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-trusted-ca-bundle\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.823605 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.823571 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-oauth-serving-cert\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.823657 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.823637 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-service-ca\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.825049 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.825028 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-serving-cert\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.825145 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.825075 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-oauth-config\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.830610 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.830591 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2ktw\" (UniqueName: \"kubernetes.io/projected/570bd4fa-1209-4755-a7f4-a60b048ad9a3-kube-api-access-g2ktw\") pod \"console-64f8b8b4f7-rhzk9\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.858628 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.858608 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:47.985088 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:47.985068 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64f8b8b4f7-rhzk9"] Jul 22 03:36:47.986847 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:36:47.986823 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod570bd4fa_1209_4755_a7f4_a60b048ad9a3.slice/crio-33c1097426b712bd5d9ec4407b665f9d0b426f0547474269490e295c32a114f8 WatchSource:0}: Error finding container 33c1097426b712bd5d9ec4407b665f9d0b426f0547474269490e295c32a114f8: Status 404 returned error can't find the container with id 33c1097426b712bd5d9ec4407b665f9d0b426f0547474269490e295c32a114f8 Jul 22 03:36:48.550188 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:48.550153 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f8b8b4f7-rhzk9" event={"ID":"570bd4fa-1209-4755-a7f4-a60b048ad9a3","Type":"ContainerStarted","Data":"b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e"} Jul 22 03:36:48.550387 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:48.550207 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f8b8b4f7-rhzk9" event={"ID":"570bd4fa-1209-4755-a7f4-a60b048ad9a3","Type":"ContainerStarted","Data":"33c1097426b712bd5d9ec4407b665f9d0b426f0547474269490e295c32a114f8"} Jul 22 03:36:48.594290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:48.594235 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64f8b8b4f7-rhzk9" podStartSLOduration=1.594219665 podStartE2EDuration="1.594219665s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:36:48.569355317 +0000 UTC m=+195.362843810" watchObservedRunningTime="2026-07-22 03:36:48.594219665 +0000 UTC m=+195.387708154" Jul 22 03:36:57.859000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:57.858965 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:57.859481 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:57.859024 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:57.863647 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:57.863623 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:58.590921 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:58.590894 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:36:58.648722 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:36:58.648696 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b599978-jj8mr"] Jul 22 03:37:13.569565 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.569526 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-584f576d4c-v4htn" podUID="9846fa3b-5928-4ac6-9100-d9633e2c0c9b" containerName="console" containerID="cri-o://7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9" gracePeriod=15 Jul 22 03:37:13.812195 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.812174 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:37:13.915546 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915462 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-oauth-serving-cert\") pod \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " Jul 22 03:37:13.915546 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915520 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-service-ca\") pod \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " Jul 22 03:37:13.915747 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915566 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-config\") pod \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " Jul 22 03:37:13.915747 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915622 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-trusted-ca-bundle\") pod \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " Jul 22 03:37:13.915747 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915646 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-serving-cert\") pod \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " Jul 22 03:37:13.915747 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915671 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfpmn\" (UniqueName: \"kubernetes.io/projected/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-kube-api-access-qfpmn\") pod \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " Jul 22 03:37:13.915747 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915692 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-oauth-config\") pod \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\" (UID: \"9846fa3b-5928-4ac6-9100-d9633e2c0c9b\") " Jul 22 03:37:13.916056 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915866 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-service-ca" (OuterVolumeSpecName: "service-ca") pod "9846fa3b-5928-4ac6-9100-d9633e2c0c9b" (UID: "9846fa3b-5928-4ac6-9100-d9633e2c0c9b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:13.916056 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.915991 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9846fa3b-5928-4ac6-9100-d9633e2c0c9b" (UID: "9846fa3b-5928-4ac6-9100-d9633e2c0c9b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:13.916172 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.916069 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-config" (OuterVolumeSpecName: "console-config") pod "9846fa3b-5928-4ac6-9100-d9633e2c0c9b" (UID: "9846fa3b-5928-4ac6-9100-d9633e2c0c9b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:13.916172 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.916090 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9846fa3b-5928-4ac6-9100-d9633e2c0c9b" (UID: "9846fa3b-5928-4ac6-9100-d9633e2c0c9b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:13.917918 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.917895 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9846fa3b-5928-4ac6-9100-d9633e2c0c9b" (UID: "9846fa3b-5928-4ac6-9100-d9633e2c0c9b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:37:13.918016 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.917955 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-kube-api-access-qfpmn" (OuterVolumeSpecName: "kube-api-access-qfpmn") pod "9846fa3b-5928-4ac6-9100-d9633e2c0c9b" (UID: "9846fa3b-5928-4ac6-9100-d9633e2c0c9b"). InnerVolumeSpecName "kube-api-access-qfpmn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:37:13.918016 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:13.917977 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9846fa3b-5928-4ac6-9100-d9633e2c0c9b" (UID: "9846fa3b-5928-4ac6-9100-d9633e2c0c9b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:37:14.016353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.016305 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-oauth-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:14.016353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.016355 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:14.016535 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.016365 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:14.016535 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.016373 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-trusted-ca-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:14.016535 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.016381 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:14.016535 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.016388 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfpmn\" (UniqueName: \"kubernetes.io/projected/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-kube-api-access-qfpmn\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:14.016535 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.016397 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9846fa3b-5928-4ac6-9100-d9633e2c0c9b-console-oauth-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:14.647472 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.647441 2564 generic.go:356] "Generic (PLEG): container finished" podID="9846fa3b-5928-4ac6-9100-d9633e2c0c9b" containerID="7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9" exitCode=0 Jul 22 03:37:14.647872 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.647489 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584f576d4c-v4htn" Jul 22 03:37:14.647872 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.647495 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584f576d4c-v4htn" event={"ID":"9846fa3b-5928-4ac6-9100-d9633e2c0c9b","Type":"ContainerDied","Data":"7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9"} Jul 22 03:37:14.647872 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.647536 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584f576d4c-v4htn" event={"ID":"9846fa3b-5928-4ac6-9100-d9633e2c0c9b","Type":"ContainerDied","Data":"02a5ea1f84ca3fc6663110a2964e16d1582ddf9bbd6f2359fb6d5a01a1fa6620"} Jul 22 03:37:14.647872 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.647556 2564 scope.go:117] "RemoveContainer" containerID="7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9" Jul 22 03:37:14.656051 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.656036 2564 scope.go:117] "RemoveContainer" containerID="7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9" Jul 22 03:37:14.656282 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:37:14.656264 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9\": container with ID starting with 7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9 not found: ID does not exist" containerID="7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9" Jul 22 03:37:14.656359 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.656288 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9"} err="failed to get container status \"7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9\": rpc error: code = NotFound desc = could not find container \"7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9\": container with ID starting with 7eb47c02949f8808b53b7222b7fb1ccdcba8bd4644bd3d04b26f5a97ffbd78d9 not found: ID does not exist" Jul 22 03:37:14.668896 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.668870 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-584f576d4c-v4htn"] Jul 22 03:37:14.672755 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:14.672733 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-584f576d4c-v4htn"] Jul 22 03:37:15.743910 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:15.743877 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9846fa3b-5928-4ac6-9100-d9633e2c0c9b" path="/var/lib/kubelet/pods/9846fa3b-5928-4ac6-9100-d9633e2c0c9b/volumes" Jul 22 03:37:23.669924 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.669877 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-8b599978-jj8mr" podUID="e845ea4a-9a7b-45f0-a08e-0bb8805a9365" containerName="console" containerID="cri-o://67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639" gracePeriod=15 Jul 22 03:37:23.915651 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.915627 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:37:23.997906 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.997872 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xpnb6\" (UniqueName: \"kubernetes.io/projected/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-kube-api-access-xpnb6\") pod \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " Jul 22 03:37:23.998096 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.997910 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-oauth-config\") pod \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " Jul 22 03:37:23.998096 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.997937 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-config\") pod \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " Jul 22 03:37:23.998096 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998018 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-service-ca\") pod \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " Jul 22 03:37:23.998096 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998059 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-oauth-serving-cert\") pod \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " Jul 22 03:37:23.998096 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998082 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-trusted-ca-bundle\") pod \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " Jul 22 03:37:23.998395 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998122 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-serving-cert\") pod \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\" (UID: \"e845ea4a-9a7b-45f0-a08e-0bb8805a9365\") " Jul 22 03:37:23.998472 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998444 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-service-ca" (OuterVolumeSpecName: "service-ca") pod "e845ea4a-9a7b-45f0-a08e-0bb8805a9365" (UID: "e845ea4a-9a7b-45f0-a08e-0bb8805a9365"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:23.998472 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998456 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-config" (OuterVolumeSpecName: "console-config") pod "e845ea4a-9a7b-45f0-a08e-0bb8805a9365" (UID: "e845ea4a-9a7b-45f0-a08e-0bb8805a9365"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:23.998569 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998532 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e845ea4a-9a7b-45f0-a08e-0bb8805a9365" (UID: "e845ea4a-9a7b-45f0-a08e-0bb8805a9365"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:23.998569 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:23.998551 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e845ea4a-9a7b-45f0-a08e-0bb8805a9365" (UID: "e845ea4a-9a7b-45f0-a08e-0bb8805a9365"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:37:24.000282 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.000256 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e845ea4a-9a7b-45f0-a08e-0bb8805a9365" (UID: "e845ea4a-9a7b-45f0-a08e-0bb8805a9365"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:37:24.000282 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.000257 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-kube-api-access-xpnb6" (OuterVolumeSpecName: "kube-api-access-xpnb6") pod "e845ea4a-9a7b-45f0-a08e-0bb8805a9365" (UID: "e845ea4a-9a7b-45f0-a08e-0bb8805a9365"). InnerVolumeSpecName "kube-api-access-xpnb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:37:24.000488 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.000445 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e845ea4a-9a7b-45f0-a08e-0bb8805a9365" (UID: "e845ea4a-9a7b-45f0-a08e-0bb8805a9365"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:37:24.098940 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.098897 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:24.098940 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.098931 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-oauth-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:24.098940 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.098943 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-trusted-ca-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:24.099180 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.098954 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:24.099180 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.098965 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xpnb6\" (UniqueName: \"kubernetes.io/projected/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-kube-api-access-xpnb6\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:24.099180 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.098973 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-oauth-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:24.099180 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.098982 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e845ea4a-9a7b-45f0-a08e-0bb8805a9365-console-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:24.687755 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.687720 2564 generic.go:356] "Generic (PLEG): container finished" podID="e845ea4a-9a7b-45f0-a08e-0bb8805a9365" containerID="67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639" exitCode=0 Jul 22 03:37:24.688144 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.687762 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8b599978-jj8mr" Jul 22 03:37:24.688144 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.687764 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b599978-jj8mr" event={"ID":"e845ea4a-9a7b-45f0-a08e-0bb8805a9365","Type":"ContainerDied","Data":"67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639"} Jul 22 03:37:24.688144 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.687895 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8b599978-jj8mr" event={"ID":"e845ea4a-9a7b-45f0-a08e-0bb8805a9365","Type":"ContainerDied","Data":"8abe772edf106d2ce3222cb60c59ba358e80a540772714d847adb54423cd90f4"} Jul 22 03:37:24.688144 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.687921 2564 scope.go:117] "RemoveContainer" containerID="67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639" Jul 22 03:37:24.696724 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.696702 2564 scope.go:117] "RemoveContainer" containerID="67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639" Jul 22 03:37:24.697028 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:37:24.697010 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639\": container with ID starting with 67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639 not found: ID does not exist" containerID="67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639" Jul 22 03:37:24.697065 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.697039 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639"} err="failed to get container status \"67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639\": rpc error: code = NotFound desc = could not find container \"67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639\": container with ID starting with 67794a5c240ce2118f925e0cd25cf697c18648fdf05d4b39690c320101e70639 not found: ID does not exist" Jul 22 03:37:24.708701 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.708672 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8b599978-jj8mr"] Jul 22 03:37:24.712214 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:24.712192 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8b599978-jj8mr"] Jul 22 03:37:25.743754 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:25.743718 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e845ea4a-9a7b-45f0-a08e-0bb8805a9365" path="/var/lib/kubelet/pods/e845ea4a-9a7b-45f0-a08e-0bb8805a9365/volumes" Jul 22 03:37:39.357873 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.357840 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf"] Jul 22 03:37:39.358275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.358139 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e845ea4a-9a7b-45f0-a08e-0bb8805a9365" containerName="console" Jul 22 03:37:39.358275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.358150 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e845ea4a-9a7b-45f0-a08e-0bb8805a9365" containerName="console" Jul 22 03:37:39.358275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.358172 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9846fa3b-5928-4ac6-9100-d9633e2c0c9b" containerName="console" Jul 22 03:37:39.358275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.358178 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9846fa3b-5928-4ac6-9100-d9633e2c0c9b" containerName="console" Jul 22 03:37:39.358275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.358225 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9846fa3b-5928-4ac6-9100-d9633e2c0c9b" containerName="console" Jul 22 03:37:39.358275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.358234 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e845ea4a-9a7b-45f0-a08e-0bb8805a9365" containerName="console" Jul 22 03:37:39.362883 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.362864 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.365913 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.365887 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zqhrx\"" Jul 22 03:37:39.365913 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.365901 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 03:37:39.366080 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.366004 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 03:37:39.370064 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.369781 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf"] Jul 22 03:37:39.513398 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.513354 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.513537 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.513421 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.513537 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.513490 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k99z5\" (UniqueName: \"kubernetes.io/projected/f5cae53e-278a-4772-99b4-7e14bb7896a7-kube-api-access-k99z5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.614796 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.614713 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k99z5\" (UniqueName: \"kubernetes.io/projected/f5cae53e-278a-4772-99b4-7e14bb7896a7-kube-api-access-k99z5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.614930 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.614809 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.614930 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.614880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.615227 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.615204 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.615300 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.615239 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.623750 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.623725 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k99z5\" (UniqueName: \"kubernetes.io/projected/f5cae53e-278a-4772-99b4-7e14bb7896a7-kube-api-access-k99z5\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.672643 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.672618 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:39.807982 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:39.807955 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf"] Jul 22 03:37:39.811515 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:37:39.811486 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5cae53e_278a_4772_99b4_7e14bb7896a7.slice/crio-d708404f43f658c9d8152ab898b3382838f12a5d0233274af61fb0c59ff007a3 WatchSource:0}: Error finding container d708404f43f658c9d8152ab898b3382838f12a5d0233274af61fb0c59ff007a3: Status 404 returned error can't find the container with id d708404f43f658c9d8152ab898b3382838f12a5d0233274af61fb0c59ff007a3 Jul 22 03:37:40.753768 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:40.753724 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" event={"ID":"f5cae53e-278a-4772-99b4-7e14bb7896a7","Type":"ContainerStarted","Data":"d708404f43f658c9d8152ab898b3382838f12a5d0233274af61fb0c59ff007a3"} Jul 22 03:37:46.781844 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:46.781791 2564 generic.go:356] "Generic (PLEG): container finished" podID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerID="162e6d0d0d483307009d4f83d4643a398ad1cdd458cbbf5374a115749c4d7f6d" exitCode=0 Jul 22 03:37:46.782256 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:46.781875 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" event={"ID":"f5cae53e-278a-4772-99b4-7e14bb7896a7","Type":"ContainerDied","Data":"162e6d0d0d483307009d4f83d4643a398ad1cdd458cbbf5374a115749c4d7f6d"} Jul 22 03:37:49.796206 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:49.796172 2564 generic.go:356] "Generic (PLEG): container finished" podID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerID="53919fff606fdfcac44375b39cecc136c6e58eeb12ab106f1a4909d3e7413823" exitCode=0 Jul 22 03:37:49.796707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:49.796222 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" event={"ID":"f5cae53e-278a-4772-99b4-7e14bb7896a7","Type":"ContainerDied","Data":"53919fff606fdfcac44375b39cecc136c6e58eeb12ab106f1a4909d3e7413823"} Jul 22 03:37:57.831628 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:57.831590 2564 generic.go:356] "Generic (PLEG): container finished" podID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerID="d8bbdbfd77f19e23994fabfd316526789dca3a88e29144c34dacdc6c1360e58b" exitCode=0 Jul 22 03:37:57.832028 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:57.831680 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" event={"ID":"f5cae53e-278a-4772-99b4-7e14bb7896a7","Type":"ContainerDied","Data":"d8bbdbfd77f19e23994fabfd316526789dca3a88e29144c34dacdc6c1360e58b"} Jul 22 03:37:58.959400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:58.959379 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:59.063202 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.063177 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-util\") pod \"f5cae53e-278a-4772-99b4-7e14bb7896a7\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " Jul 22 03:37:59.063311 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.063250 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-bundle\") pod \"f5cae53e-278a-4772-99b4-7e14bb7896a7\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " Jul 22 03:37:59.063311 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.063279 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k99z5\" (UniqueName: \"kubernetes.io/projected/f5cae53e-278a-4772-99b4-7e14bb7896a7-kube-api-access-k99z5\") pod \"f5cae53e-278a-4772-99b4-7e14bb7896a7\" (UID: \"f5cae53e-278a-4772-99b4-7e14bb7896a7\") " Jul 22 03:37:59.063996 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.063965 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-bundle" (OuterVolumeSpecName: "bundle") pod "f5cae53e-278a-4772-99b4-7e14bb7896a7" (UID: "f5cae53e-278a-4772-99b4-7e14bb7896a7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:37:59.065644 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.065625 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5cae53e-278a-4772-99b4-7e14bb7896a7-kube-api-access-k99z5" (OuterVolumeSpecName: "kube-api-access-k99z5") pod "f5cae53e-278a-4772-99b4-7e14bb7896a7" (UID: "f5cae53e-278a-4772-99b4-7e14bb7896a7"). InnerVolumeSpecName "kube-api-access-k99z5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:37:59.067442 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.067420 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-util" (OuterVolumeSpecName: "util") pod "f5cae53e-278a-4772-99b4-7e14bb7896a7" (UID: "f5cae53e-278a-4772-99b4-7e14bb7896a7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:37:59.163889 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.163829 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-util\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:59.163889 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.163850 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5cae53e-278a-4772-99b4-7e14bb7896a7-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:59.163889 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.163858 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k99z5\" (UniqueName: \"kubernetes.io/projected/f5cae53e-278a-4772-99b4-7e14bb7896a7-kube-api-access-k99z5\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:37:59.840921 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.840889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" event={"ID":"f5cae53e-278a-4772-99b4-7e14bb7896a7","Type":"ContainerDied","Data":"d708404f43f658c9d8152ab898b3382838f12a5d0233274af61fb0c59ff007a3"} Jul 22 03:37:59.840921 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.840915 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540jmkmf" Jul 22 03:37:59.841079 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:37:59.840932 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d708404f43f658c9d8152ab898b3382838f12a5d0233274af61fb0c59ff007a3" Jul 22 03:38:12.255595 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.255554 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69"] Jul 22 03:38:12.256070 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.255905 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerName="extract" Jul 22 03:38:12.256070 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.255921 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerName="extract" Jul 22 03:38:12.256070 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.255946 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerName="util" Jul 22 03:38:12.256070 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.255951 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerName="util" Jul 22 03:38:12.256070 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.255958 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerName="pull" Jul 22 03:38:12.256070 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.255963 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerName="pull" Jul 22 03:38:12.256070 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.256021 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5cae53e-278a-4772-99b4-7e14bb7896a7" containerName="extract" Jul 22 03:38:12.259867 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.259842 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.264260 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.264237 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zqhrx\"" Jul 22 03:38:12.264542 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.264525 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 03:38:12.265539 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.265519 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 03:38:12.272323 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.272295 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69"] Jul 22 03:38:12.356492 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.356467 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7jx2\" (UniqueName: \"kubernetes.io/projected/82173f85-045e-4e6f-9065-1b064f54893a-kube-api-access-t7jx2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.356589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.356499 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.356589 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.356538 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.457826 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.457799 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7jx2\" (UniqueName: \"kubernetes.io/projected/82173f85-045e-4e6f-9065-1b064f54893a-kube-api-access-t7jx2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.457939 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.457833 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.458011 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.457984 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.458254 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.458230 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.458304 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.458287 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.468550 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.468518 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7jx2\" (UniqueName: \"kubernetes.io/projected/82173f85-045e-4e6f-9065-1b064f54893a-kube-api-access-t7jx2\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.570317 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.570258 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:12.711280 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.711254 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69"] Jul 22 03:38:12.713243 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:38:12.713204 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82173f85_045e_4e6f_9065_1b064f54893a.slice/crio-05016466908ec268b0a0459f80f434587a13d318969fc61fe14a9b8cfb38aa74 WatchSource:0}: Error finding container 05016466908ec268b0a0459f80f434587a13d318969fc61fe14a9b8cfb38aa74: Status 404 returned error can't find the container with id 05016466908ec268b0a0459f80f434587a13d318969fc61fe14a9b8cfb38aa74 Jul 22 03:38:12.894780 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.894717 2564 generic.go:356] "Generic (PLEG): container finished" podID="82173f85-045e-4e6f-9065-1b064f54893a" containerID="d0ec6b1d0cb7b2c176227a6565f5110e6d2d1418796f11de702eaca12b2c8802" exitCode=0 Jul 22 03:38:12.894870 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.894787 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" event={"ID":"82173f85-045e-4e6f-9065-1b064f54893a","Type":"ContainerDied","Data":"d0ec6b1d0cb7b2c176227a6565f5110e6d2d1418796f11de702eaca12b2c8802"} Jul 22 03:38:12.894870 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:12.894831 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" event={"ID":"82173f85-045e-4e6f-9065-1b064f54893a","Type":"ContainerStarted","Data":"05016466908ec268b0a0459f80f434587a13d318969fc61fe14a9b8cfb38aa74"} Jul 22 03:38:13.968912 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:13.968870 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kqswr"] Jul 22 03:38:13.975245 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:13.975225 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:13.978290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:13.978190 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-tfdmn\"" Jul 22 03:38:13.978290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:13.978283 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 03:38:13.978474 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:13.978327 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 03:38:13.981383 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:13.981360 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kqswr"] Jul 22 03:38:14.072259 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.072228 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5be4ea7f-5b7b-4d96-baab-f74bac73d18b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kqswr\" (UID: \"5be4ea7f-5b7b-4d96-baab-f74bac73d18b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:14.072409 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.072295 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l26ck\" (UniqueName: \"kubernetes.io/projected/5be4ea7f-5b7b-4d96-baab-f74bac73d18b-kube-api-access-l26ck\") pod \"cert-manager-webhook-6486ffd997-kqswr\" (UID: \"5be4ea7f-5b7b-4d96-baab-f74bac73d18b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:14.173531 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.173505 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5be4ea7f-5b7b-4d96-baab-f74bac73d18b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kqswr\" (UID: \"5be4ea7f-5b7b-4d96-baab-f74bac73d18b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:14.173687 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.173558 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l26ck\" (UniqueName: \"kubernetes.io/projected/5be4ea7f-5b7b-4d96-baab-f74bac73d18b-kube-api-access-l26ck\") pod \"cert-manager-webhook-6486ffd997-kqswr\" (UID: \"5be4ea7f-5b7b-4d96-baab-f74bac73d18b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:14.183027 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.182998 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l26ck\" (UniqueName: \"kubernetes.io/projected/5be4ea7f-5b7b-4d96-baab-f74bac73d18b-kube-api-access-l26ck\") pod \"cert-manager-webhook-6486ffd997-kqswr\" (UID: \"5be4ea7f-5b7b-4d96-baab-f74bac73d18b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:14.183162 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.183078 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/5be4ea7f-5b7b-4d96-baab-f74bac73d18b-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-kqswr\" (UID: \"5be4ea7f-5b7b-4d96-baab-f74bac73d18b\") " pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:14.286487 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.286458 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:14.609679 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.609652 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-kqswr"] Jul 22 03:38:14.613025 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:38:14.612996 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5be4ea7f_5b7b_4d96_baab_f74bac73d18b.slice/crio-17420dfb09ffd42876443a5cf7d99ce96fdceef222adf6f85d4c1df063be3553 WatchSource:0}: Error finding container 17420dfb09ffd42876443a5cf7d99ce96fdceef222adf6f85d4c1df063be3553: Status 404 returned error can't find the container with id 17420dfb09ffd42876443a5cf7d99ce96fdceef222adf6f85d4c1df063be3553 Jul 22 03:38:14.904803 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.904739 2564 generic.go:356] "Generic (PLEG): container finished" podID="82173f85-045e-4e6f-9065-1b064f54893a" containerID="37e3c0cd7e0d5e7dc1fcf17ca3a748142e5990681084f272f38026ce534f5ab0" exitCode=0 Jul 22 03:38:14.904912 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.904835 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" event={"ID":"82173f85-045e-4e6f-9065-1b064f54893a","Type":"ContainerDied","Data":"37e3c0cd7e0d5e7dc1fcf17ca3a748142e5990681084f272f38026ce534f5ab0"} Jul 22 03:38:14.906111 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:14.906088 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" event={"ID":"5be4ea7f-5b7b-4d96-baab-f74bac73d18b","Type":"ContainerStarted","Data":"17420dfb09ffd42876443a5cf7d99ce96fdceef222adf6f85d4c1df063be3553"} Jul 22 03:38:15.913844 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:15.913812 2564 generic.go:356] "Generic (PLEG): container finished" podID="82173f85-045e-4e6f-9065-1b064f54893a" containerID="f5dd67f94fa5f77710d0450fd282f17def82c12c7d5270347c79c54a2771e150" exitCode=0 Jul 22 03:38:15.914241 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:15.913889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" event={"ID":"82173f85-045e-4e6f-9065-1b064f54893a","Type":"ContainerDied","Data":"f5dd67f94fa5f77710d0450fd282f17def82c12c7d5270347c79c54a2771e150"} Jul 22 03:38:17.204939 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.204913 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:17.300891 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.300867 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-util\") pod \"82173f85-045e-4e6f-9065-1b064f54893a\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " Jul 22 03:38:17.301011 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.300940 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7jx2\" (UniqueName: \"kubernetes.io/projected/82173f85-045e-4e6f-9065-1b064f54893a-kube-api-access-t7jx2\") pod \"82173f85-045e-4e6f-9065-1b064f54893a\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " Jul 22 03:38:17.303045 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.302958 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82173f85-045e-4e6f-9065-1b064f54893a-kube-api-access-t7jx2" (OuterVolumeSpecName: "kube-api-access-t7jx2") pod "82173f85-045e-4e6f-9065-1b064f54893a" (UID: "82173f85-045e-4e6f-9065-1b064f54893a"). InnerVolumeSpecName "kube-api-access-t7jx2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:38:17.306636 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.306608 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-util" (OuterVolumeSpecName: "util") pod "82173f85-045e-4e6f-9065-1b064f54893a" (UID: "82173f85-045e-4e6f-9065-1b064f54893a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:17.401301 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.401279 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-bundle\") pod \"82173f85-045e-4e6f-9065-1b064f54893a\" (UID: \"82173f85-045e-4e6f-9065-1b064f54893a\") " Jul 22 03:38:17.401459 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.401447 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7jx2\" (UniqueName: \"kubernetes.io/projected/82173f85-045e-4e6f-9065-1b064f54893a-kube-api-access-t7jx2\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:38:17.401502 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.401462 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-util\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:38:17.401599 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.401579 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-bundle" (OuterVolumeSpecName: "bundle") pod "82173f85-045e-4e6f-9065-1b064f54893a" (UID: "82173f85-045e-4e6f-9065-1b064f54893a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:17.502306 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.502286 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/82173f85-045e-4e6f-9065-1b064f54893a-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:38:17.925019 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.924999 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" Jul 22 03:38:17.925139 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.925029 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87frcv69" event={"ID":"82173f85-045e-4e6f-9065-1b064f54893a","Type":"ContainerDied","Data":"05016466908ec268b0a0459f80f434587a13d318969fc61fe14a9b8cfb38aa74"} Jul 22 03:38:17.925139 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.925075 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="05016466908ec268b0a0459f80f434587a13d318969fc61fe14a9b8cfb38aa74" Jul 22 03:38:17.926748 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.926706 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" event={"ID":"5be4ea7f-5b7b-4d96-baab-f74bac73d18b","Type":"ContainerStarted","Data":"cd674a50f01dff58dff57701020329acf1301d8456a38754359a3cb2d7bf1211"} Jul 22 03:38:17.926856 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.926802 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:17.946958 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:17.946916 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" podStartSLOduration=2.340289629 podStartE2EDuration="4.946902903s" totalImagesPullingTime="2.606613274s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:38:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:38:17.945388138 +0000 UTC m=+284.738876630" watchObservedRunningTime="2026-07-22 03:38:17.946902903 +0000 UTC m=+284.740391394" Jul 22 03:38:21.911517 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911481 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h9r2w"] Jul 22 03:38:21.911875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911788 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82173f85-045e-4e6f-9065-1b064f54893a" containerName="extract" Jul 22 03:38:21.911875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911800 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="82173f85-045e-4e6f-9065-1b064f54893a" containerName="extract" Jul 22 03:38:21.911875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911824 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82173f85-045e-4e6f-9065-1b064f54893a" containerName="util" Jul 22 03:38:21.911875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911830 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="82173f85-045e-4e6f-9065-1b064f54893a" containerName="util" Jul 22 03:38:21.911875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911836 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82173f85-045e-4e6f-9065-1b064f54893a" containerName="pull" Jul 22 03:38:21.911875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911841 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="82173f85-045e-4e6f-9065-1b064f54893a" containerName="pull" Jul 22 03:38:21.912044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.911900 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="82173f85-045e-4e6f-9065-1b064f54893a" containerName="extract" Jul 22 03:38:21.914078 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.914062 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:21.917056 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.917039 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-xrcnk\"" Jul 22 03:38:21.925701 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.925676 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h9r2w"] Jul 22 03:38:21.930352 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.930316 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zghj7\" (UniqueName: \"kubernetes.io/projected/dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc-kube-api-access-zghj7\") pod \"cert-manager-55b4d954d5-h9r2w\" (UID: \"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc\") " pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:21.930471 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:21.930447 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc-bound-sa-token\") pod \"cert-manager-55b4d954d5-h9r2w\" (UID: \"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc\") " pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:22.031449 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.031428 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zghj7\" (UniqueName: \"kubernetes.io/projected/dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc-kube-api-access-zghj7\") pod \"cert-manager-55b4d954d5-h9r2w\" (UID: \"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc\") " pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:22.031541 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.031488 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc-bound-sa-token\") pod \"cert-manager-55b4d954d5-h9r2w\" (UID: \"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc\") " pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:22.040550 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.040531 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc-bound-sa-token\") pod \"cert-manager-55b4d954d5-h9r2w\" (UID: \"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc\") " pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:22.040826 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.040804 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zghj7\" (UniqueName: \"kubernetes.io/projected/dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc-kube-api-access-zghj7\") pod \"cert-manager-55b4d954d5-h9r2w\" (UID: \"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc\") " pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:22.222932 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.222880 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-h9r2w" Jul 22 03:38:22.365696 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.365661 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-h9r2w"] Jul 22 03:38:22.369033 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:38:22.369007 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc21ad4b_fa68_43c3_b198_0c4f9f0ef7dc.slice/crio-aabd9885a4af6076d3e3f61ccf31eff9a5c76c915e9f403ad4822a85c352b140 WatchSource:0}: Error finding container aabd9885a4af6076d3e3f61ccf31eff9a5c76c915e9f403ad4822a85c352b140: Status 404 returned error can't find the container with id aabd9885a4af6076d3e3f61ccf31eff9a5c76c915e9f403ad4822a85c352b140 Jul 22 03:38:22.950059 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.950023 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h9r2w" event={"ID":"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc","Type":"ContainerStarted","Data":"6e7e3f416c9f73385fedfa0a041a440c9f943aba5290efbd79c9a7b7614ea634"} Jul 22 03:38:22.950059 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.950064 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-h9r2w" event={"ID":"dc21ad4b-fa68-43c3-b198-0c4f9f0ef7dc","Type":"ContainerStarted","Data":"aabd9885a4af6076d3e3f61ccf31eff9a5c76c915e9f403ad4822a85c352b140"} Jul 22 03:38:22.969626 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:22.969586 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-h9r2w" podStartSLOduration=1.969571621 podStartE2EDuration="1.969571621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:38:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:38:22.968251055 +0000 UTC m=+289.761739551" watchObservedRunningTime="2026-07-22 03:38:22.969571621 +0000 UTC m=+289.763060113" Jul 22 03:38:23.934299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:23.934266 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-kqswr" Jul 22 03:38:32.193212 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.193175 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt"] Jul 22 03:38:32.195735 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.195718 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.198781 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.198760 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 03:38:32.198918 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.198776 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 03:38:32.200068 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.200050 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-zqhrx\"" Jul 22 03:38:32.206851 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.206822 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt"] Jul 22 03:38:32.302512 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.302472 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.302701 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.302541 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.302701 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.302577 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kswwt\" (UniqueName: \"kubernetes.io/projected/c9107d2a-14e2-4244-9504-1b292087a330-kube-api-access-kswwt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.403411 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.403368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.403617 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.403442 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.403617 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.403478 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kswwt\" (UniqueName: \"kubernetes.io/projected/c9107d2a-14e2-4244-9504-1b292087a330-kube-api-access-kswwt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.403753 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.403728 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.403847 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.403828 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.414366 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.414319 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kswwt\" (UniqueName: \"kubernetes.io/projected/c9107d2a-14e2-4244-9504-1b292087a330-kube-api-access-kswwt\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.505468 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.505432 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:32.648972 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.648942 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt"] Jul 22 03:38:32.650930 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:38:32.650889 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9107d2a_14e2_4244_9504_1b292087a330.slice/crio-16c6229ba11115f3e7eb1f15d02962baadb7a32d3931289d551c516cb29b3332 WatchSource:0}: Error finding container 16c6229ba11115f3e7eb1f15d02962baadb7a32d3931289d551c516cb29b3332: Status 404 returned error can't find the container with id 16c6229ba11115f3e7eb1f15d02962baadb7a32d3931289d551c516cb29b3332 Jul 22 03:38:32.997631 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.997591 2564 generic.go:356] "Generic (PLEG): container finished" podID="c9107d2a-14e2-4244-9504-1b292087a330" containerID="a9042d0bd6f76b33041f8bd108d3944e050cdb21f087fc28212a9f3c771bf36b" exitCode=0 Jul 22 03:38:32.997823 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.997724 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" event={"ID":"c9107d2a-14e2-4244-9504-1b292087a330","Type":"ContainerDied","Data":"a9042d0bd6f76b33041f8bd108d3944e050cdb21f087fc28212a9f3c771bf36b"} Jul 22 03:38:32.997823 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:32.997813 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" event={"ID":"c9107d2a-14e2-4244-9504-1b292087a330","Type":"ContainerStarted","Data":"16c6229ba11115f3e7eb1f15d02962baadb7a32d3931289d551c516cb29b3332"} Jul 22 03:38:33.604970 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:33.604948 2564 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 03:38:34.004893 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:34.004860 2564 generic.go:356] "Generic (PLEG): container finished" podID="c9107d2a-14e2-4244-9504-1b292087a330" containerID="c79574d74c4fe8a542e1555003ea002bb444f638d8cd9f03a0bdc98982e7524f" exitCode=0 Jul 22 03:38:34.005997 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:34.004900 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" event={"ID":"c9107d2a-14e2-4244-9504-1b292087a330","Type":"ContainerDied","Data":"c79574d74c4fe8a542e1555003ea002bb444f638d8cd9f03a0bdc98982e7524f"} Jul 22 03:38:35.013582 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:35.013534 2564 generic.go:356] "Generic (PLEG): container finished" podID="c9107d2a-14e2-4244-9504-1b292087a330" containerID="37133b6535307ffc4d1d76bd6036bf1d9b30eb27656e27ba7dc409da82fc5a59" exitCode=0 Jul 22 03:38:35.014012 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:35.013634 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" event={"ID":"c9107d2a-14e2-4244-9504-1b292087a330","Type":"ContainerDied","Data":"37133b6535307ffc4d1d76bd6036bf1d9b30eb27656e27ba7dc409da82fc5a59"} Jul 22 03:38:36.151294 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.151272 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:36.234754 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.234727 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kswwt\" (UniqueName: \"kubernetes.io/projected/c9107d2a-14e2-4244-9504-1b292087a330-kube-api-access-kswwt\") pod \"c9107d2a-14e2-4244-9504-1b292087a330\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " Jul 22 03:38:36.234890 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.234802 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-util\") pod \"c9107d2a-14e2-4244-9504-1b292087a330\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " Jul 22 03:38:36.234890 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.234826 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-bundle\") pod \"c9107d2a-14e2-4244-9504-1b292087a330\" (UID: \"c9107d2a-14e2-4244-9504-1b292087a330\") " Jul 22 03:38:36.235566 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.235540 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-bundle" (OuterVolumeSpecName: "bundle") pod "c9107d2a-14e2-4244-9504-1b292087a330" (UID: "c9107d2a-14e2-4244-9504-1b292087a330"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:36.236894 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.236869 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9107d2a-14e2-4244-9504-1b292087a330-kube-api-access-kswwt" (OuterVolumeSpecName: "kube-api-access-kswwt") pod "c9107d2a-14e2-4244-9504-1b292087a330" (UID: "c9107d2a-14e2-4244-9504-1b292087a330"). InnerVolumeSpecName "kube-api-access-kswwt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:38:36.240371 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.240325 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-util" (OuterVolumeSpecName: "util") pod "c9107d2a-14e2-4244-9504-1b292087a330" (UID: "c9107d2a-14e2-4244-9504-1b292087a330"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:38:36.336040 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.335989 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-util\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:38:36.336040 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.336012 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9107d2a-14e2-4244-9504-1b292087a330-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:38:36.336040 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:36.336021 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kswwt\" (UniqueName: \"kubernetes.io/projected/c9107d2a-14e2-4244-9504-1b292087a330-kube-api-access-kswwt\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:38:37.025531 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:37.025501 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" Jul 22 03:38:37.025717 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:37.025530 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afblgpt" event={"ID":"c9107d2a-14e2-4244-9504-1b292087a330","Type":"ContainerDied","Data":"16c6229ba11115f3e7eb1f15d02962baadb7a32d3931289d551c516cb29b3332"} Jul 22 03:38:37.025717 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:37.025578 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="16c6229ba11115f3e7eb1f15d02962baadb7a32d3931289d551c516cb29b3332" Jul 22 03:38:43.986709 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.986663 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r"] Jul 22 03:38:43.987185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.987096 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9107d2a-14e2-4244-9504-1b292087a330" containerName="extract" Jul 22 03:38:43.987185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.987116 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9107d2a-14e2-4244-9504-1b292087a330" containerName="extract" Jul 22 03:38:43.987185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.987133 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9107d2a-14e2-4244-9504-1b292087a330" containerName="pull" Jul 22 03:38:43.987185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.987141 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9107d2a-14e2-4244-9504-1b292087a330" containerName="pull" Jul 22 03:38:43.987185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.987165 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9107d2a-14e2-4244-9504-1b292087a330" containerName="util" Jul 22 03:38:43.987185 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.987172 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9107d2a-14e2-4244-9504-1b292087a330" containerName="util" Jul 22 03:38:43.987516 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.987281 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9107d2a-14e2-4244-9504-1b292087a330" containerName="extract" Jul 22 03:38:43.989697 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.989675 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:43.997000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.996976 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 03:38:43.997137 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.996975 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 03:38:43.997137 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.996985 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 03:38:43.997137 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.997031 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-2zzk4\"" Jul 22 03:38:43.997137 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.997024 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 03:38:43.997549 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:43.997535 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 03:38:44.021255 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.021229 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r"] Jul 22 03:38:44.097933 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.097899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-cert\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.098126 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.097953 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-kube-api-access-j44ft\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.098126 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.098030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-metrics-cert\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.098126 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.098083 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-manager-config\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.199426 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.199394 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-manager-config\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.199603 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.199454 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-cert\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.199603 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.199504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-kube-api-access-j44ft\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.199603 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.199555 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-metrics-cert\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.200188 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.200113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-manager-config\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.202129 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.202108 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-metrics-cert\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.202213 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.202131 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-cert\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.209846 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.209822 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j44ft\" (UniqueName: \"kubernetes.io/projected/ec6419df-c83c-44ff-83b8-87a8e4bae5e4-kube-api-access-j44ft\") pod \"lws-controller-manager-78585bcb5c-fmc8r\" (UID: \"ec6419df-c83c-44ff-83b8-87a8e4bae5e4\") " pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.298974 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.298945 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:44.441854 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.441817 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r"] Jul 22 03:38:44.444992 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:38:44.444967 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec6419df_c83c_44ff_83b8_87a8e4bae5e4.slice/crio-80127470d4d6f6d55b7cc223dfdc8d373e1c7416400ab2f7da53024c50dccd6d WatchSource:0}: Error finding container 80127470d4d6f6d55b7cc223dfdc8d373e1c7416400ab2f7da53024c50dccd6d: Status 404 returned error can't find the container with id 80127470d4d6f6d55b7cc223dfdc8d373e1c7416400ab2f7da53024c50dccd6d Jul 22 03:38:44.446923 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.446909 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:38:44.938253 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.935620 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx"] Jul 22 03:38:44.940526 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.940494 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:44.944236 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.944210 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-qfwz9\"" Jul 22 03:38:44.944462 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.944312 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 03:38:44.944462 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.944456 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 03:38:44.944924 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.944906 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 03:38:44.945651 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.945627 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 03:38:44.978819 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:44.978791 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx"] Jul 22 03:38:45.008048 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.008014 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8xwv\" (UniqueName: \"kubernetes.io/projected/79b4fd24-783d-4058-b67c-90fb163a28a2-kube-api-access-n8xwv\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.008467 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.008076 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/79b4fd24-783d-4058-b67c-90fb163a28a2-manifests\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.008467 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.008112 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/79b4fd24-783d-4058-b67c-90fb163a28a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.008467 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.008205 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79b4fd24-783d-4058-b67c-90fb163a28a2-tmp\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.008467 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.008241 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79b4fd24-783d-4058-b67c-90fb163a28a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.064064 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.064025 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" event={"ID":"ec6419df-c83c-44ff-83b8-87a8e4bae5e4","Type":"ContainerStarted","Data":"80127470d4d6f6d55b7cc223dfdc8d373e1c7416400ab2f7da53024c50dccd6d"} Jul 22 03:38:45.109732 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.109658 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8xwv\" (UniqueName: \"kubernetes.io/projected/79b4fd24-783d-4058-b67c-90fb163a28a2-kube-api-access-n8xwv\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.109732 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.109721 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/79b4fd24-783d-4058-b67c-90fb163a28a2-manifests\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.110071 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.109758 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/79b4fd24-783d-4058-b67c-90fb163a28a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.110071 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.109802 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79b4fd24-783d-4058-b67c-90fb163a28a2-tmp\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.110071 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.109832 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79b4fd24-783d-4058-b67c-90fb163a28a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.110289 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.110184 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/79b4fd24-783d-4058-b67c-90fb163a28a2-manifests\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.112624 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.112599 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/79b4fd24-783d-4058-b67c-90fb163a28a2-tmp\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.112768 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.112749 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/79b4fd24-783d-4058-b67c-90fb163a28a2-webhook-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.112835 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.112803 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/79b4fd24-783d-4058-b67c-90fb163a28a2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.124784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.124754 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8xwv\" (UniqueName: \"kubernetes.io/projected/79b4fd24-783d-4058-b67c-90fb163a28a2-kube-api-access-n8xwv\") pod \"opendatahub-operator-controller-manager-5fd47b996b-b4xqx\" (UID: \"79b4fd24-783d-4058-b67c-90fb163a28a2\") " pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.251081 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.251047 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:45.408447 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:45.408403 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx"] Jul 22 03:38:45.411279 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:38:45.411249 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79b4fd24_783d_4058_b67c_90fb163a28a2.slice/crio-069f06056f9ed8ac0d69536e5a3f921309201670e3cb4513eb5775e70d49d467 WatchSource:0}: Error finding container 069f06056f9ed8ac0d69536e5a3f921309201670e3cb4513eb5775e70d49d467: Status 404 returned error can't find the container with id 069f06056f9ed8ac0d69536e5a3f921309201670e3cb4513eb5775e70d49d467 Jul 22 03:38:46.078690 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:46.078591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" event={"ID":"79b4fd24-783d-4058-b67c-90fb163a28a2","Type":"ContainerStarted","Data":"069f06056f9ed8ac0d69536e5a3f921309201670e3cb4513eb5775e70d49d467"} Jul 22 03:38:46.081326 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:46.080954 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" event={"ID":"ec6419df-c83c-44ff-83b8-87a8e4bae5e4","Type":"ContainerStarted","Data":"02e5cff2b657c5885ff8e3c41701017b838b43db73ce3cdc6cc17d37a497ea71"} Jul 22 03:38:46.081326 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:46.081190 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:38:46.101935 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:46.101878 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" podStartSLOduration=1.7265332820000001 podStartE2EDuration="3.101857361s" totalImagesPullingTime="1.375324079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:38:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:38:46.099674555 +0000 UTC m=+312.893163058" watchObservedRunningTime="2026-07-22 03:38:46.101857361 +0000 UTC m=+312.895345856" Jul 22 03:38:49.097720 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:49.097687 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" event={"ID":"79b4fd24-783d-4058-b67c-90fb163a28a2","Type":"ContainerStarted","Data":"73df6eb5839af7d7a68483f2af99994ada3c7c174966525658fbd16b6ff3ada1"} Jul 22 03:38:50.104087 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:50.104050 2564 generic.go:356] "Generic (PLEG): container finished" podID="79b4fd24-783d-4058-b67c-90fb163a28a2" containerID="73df6eb5839af7d7a68483f2af99994ada3c7c174966525658fbd16b6ff3ada1" exitCode=0 Jul 22 03:38:50.104545 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:50.104115 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" event={"ID":"79b4fd24-783d-4058-b67c-90fb163a28a2","Type":"ContainerDied","Data":"73df6eb5839af7d7a68483f2af99994ada3c7c174966525658fbd16b6ff3ada1"} Jul 22 03:38:51.110010 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:51.109974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" event={"ID":"79b4fd24-783d-4058-b67c-90fb163a28a2","Type":"ContainerStarted","Data":"43f1c8b3f4e68e71dc96f625f2f419d2f9ff6113b872e5bc3f905dfb5ee50088"} Jul 22 03:38:51.110469 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:51.110106 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:38:51.135150 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:51.135081 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" podStartSLOduration=2.8006338570000002 podStartE2EDuration="7.135064138s" totalImagesPullingTime="3.293736307s" totalInitContainerRuntime="1.040693974s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:38:44 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:38:51.133559355 +0000 UTC m=+317.927047846" watchObservedRunningTime="2026-07-22 03:38:51.135064138 +0000 UTC m=+317.928552632" Jul 22 03:38:57.089478 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:38:57.089447 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78585bcb5c-fmc8r" Jul 22 03:39:02.118968 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:02.118933 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5fd47b996b-b4xqx" Jul 22 03:39:17.264875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.264842 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz"] Jul 22 03:39:17.273522 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.273497 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.276643 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.276620 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 03:39:17.276784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.276760 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 03:39:17.276858 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.276809 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 03:39:17.277320 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.277297 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-n9xft\"" Jul 22 03:39:17.285534 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.285511 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz"] Jul 22 03:39:17.288516 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288496 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288614 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288523 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dszm5\" (UniqueName: \"kubernetes.io/projected/3a5870d1-e07b-48ed-bd97-ba6406dc054c-kube-api-access-dszm5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288614 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288551 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288614 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288574 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288710 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288632 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288710 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288653 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288710 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288670 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288710 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288706 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.288827 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.288748 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.389865 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.389843 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.389978 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.389870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.389978 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.389895 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.389978 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.389918 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.389978 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.389971 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390188 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.389995 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dszm5\" (UniqueName: \"kubernetes.io/projected/3a5870d1-e07b-48ed-bd97-ba6406dc054c-kube-api-access-dszm5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390188 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.390100 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390188 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.390138 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390188 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.390185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390409 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.390373 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390516 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.390417 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390738 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.390594 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.390738 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.390653 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.391078 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.391054 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.392270 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.392254 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.392611 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.392591 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.398590 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.398562 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3a5870d1-e07b-48ed-bd97-ba6406dc054c-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.398763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.398743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dszm5\" (UniqueName: \"kubernetes.io/projected/3a5870d1-e07b-48ed-bd97-ba6406dc054c-kube-api-access-dszm5\") pod \"data-science-gateway-data-science-gateway-class-6576859cf47mgtz\" (UID: \"3a5870d1-e07b-48ed-bd97-ba6406dc054c\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.584458 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.584407 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:17.735495 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:17.735467 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz"] Jul 22 03:39:17.738517 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:39:17.738491 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a5870d1_e07b_48ed_bd97_ba6406dc054c.slice/crio-0c6481a15ae539abc2925ebd21f9ea3af0ed482e41b7d168fb07d0c715e1e905 WatchSource:0}: Error finding container 0c6481a15ae539abc2925ebd21f9ea3af0ed482e41b7d168fb07d0c715e1e905: Status 404 returned error can't find the container with id 0c6481a15ae539abc2925ebd21f9ea3af0ed482e41b7d168fb07d0c715e1e905 Jul 22 03:39:18.231696 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:18.231654 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" event={"ID":"3a5870d1-e07b-48ed-bd97-ba6406dc054c","Type":"ContainerStarted","Data":"0c6481a15ae539abc2925ebd21f9ea3af0ed482e41b7d168fb07d0c715e1e905"} Jul 22 03:39:19.714363 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.714301 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j"] Jul 22 03:39:19.719486 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.719454 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.723003 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.722973 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-2dtvq\"" Jul 22 03:39:19.723572 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.723550 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 03:39:19.723950 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.723932 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 03:39:19.724254 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.724236 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 03:39:19.732555 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.732520 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j"] Jul 22 03:39:19.808791 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.808759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.808791 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.808794 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0af4091-4932-43c9-924e-9f17470da231-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.808991 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.808823 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d0af4091-4932-43c9-924e-9f17470da231-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.808991 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.808899 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.808991 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.808959 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpdr4\" (UniqueName: \"kubernetes.io/projected/d0af4091-4932-43c9-924e-9f17470da231-kube-api-access-dpdr4\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.808991 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.808987 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0af4091-4932-43c9-924e-9f17470da231-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.809178 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.809074 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.910353 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.910313 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpdr4\" (UniqueName: \"kubernetes.io/projected/d0af4091-4932-43c9-924e-9f17470da231-kube-api-access-dpdr4\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.910500 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.910359 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0af4091-4932-43c9-924e-9f17470da231-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.910500 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.910387 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.910500 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.910411 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.910500 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.910452 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0af4091-4932-43c9-924e-9f17470da231-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.910500 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.910490 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d0af4091-4932-43c9-924e-9f17470da231-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.910784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.910522 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.911491 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.911463 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/d0af4091-4932-43c9-924e-9f17470da231-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.912938 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.912915 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/d0af4091-4932-43c9-924e-9f17470da231-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.913066 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.912964 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.913066 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.913045 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.913066 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.913061 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/d0af4091-4932-43c9-924e-9f17470da231-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.922684 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.922665 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d0af4091-4932-43c9-924e-9f17470da231-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:19.925511 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:19.925488 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpdr4\" (UniqueName: \"kubernetes.io/projected/d0af4091-4932-43c9-924e-9f17470da231-kube-api-access-dpdr4\") pod \"istiod-openshift-gateway-94bb8fbfd-ck86j\" (UID: \"d0af4091-4932-43c9-924e-9f17470da231\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:20.034469 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:20.034451 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:20.169913 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:20.169890 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j"] Jul 22 03:39:20.171684 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:39:20.171663 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0af4091_4932_43c9_924e_9f17470da231.slice/crio-e0140959c2298de5eb54e78ef830c68d6d3ec9afb27b3031f1c688a456b74e99 WatchSource:0}: Error finding container e0140959c2298de5eb54e78ef830c68d6d3ec9afb27b3031f1c688a456b74e99: Status 404 returned error can't find the container with id e0140959c2298de5eb54e78ef830c68d6d3ec9afb27b3031f1c688a456b74e99 Jul 22 03:39:20.241732 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:20.241699 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" event={"ID":"d0af4091-4932-43c9-924e-9f17470da231","Type":"ContainerStarted","Data":"e0140959c2298de5eb54e78ef830c68d6d3ec9afb27b3031f1c688a456b74e99"} Jul 22 03:39:22.881047 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:22.881005 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:39:22.881401 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:22.881084 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:39:23.260042 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:23.260012 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" event={"ID":"d0af4091-4932-43c9-924e-9f17470da231","Type":"ContainerStarted","Data":"56ebe4f70b2820b2524c003f5e2eb4cb7188e2068742dfb498107550cf337dbd"} Jul 22 03:39:23.260293 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:23.260232 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:23.262354 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:23.262300 2564 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-ck86j container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 03:39:23.262473 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:23.262388 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" podUID="d0af4091-4932-43c9-924e-9f17470da231" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 03:39:23.323354 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:23.323276 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" podStartSLOduration=1.616143721 podStartE2EDuration="4.323253708s" totalImagesPullingTime="2.707109987s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:39:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:39:23.322779693 +0000 UTC m=+350.116268185" watchObservedRunningTime="2026-07-22 03:39:23.323253708 +0000 UTC m=+350.116742199" Jul 22 03:39:24.266818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:24.266740 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-ck86j" Jul 22 03:39:28.970388 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:28.970352 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:39:28.970642 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:28.970421 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:39:28.970642 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:28.970451 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:39:29.291033 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:29.291002 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" event={"ID":"3a5870d1-e07b-48ed-bd97-ba6406dc054c","Type":"ContainerStarted","Data":"4bb91f7e543bafe2894db39c652500cf7a1e4967c5ce385e28ecee0e56cad15d"} Jul 22 03:39:29.314979 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:29.314934 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" podStartSLOduration=1.085230788 podStartE2EDuration="12.314920207s" totalImagesPullingTime="11.229689419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:39:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:39:29.312225433 +0000 UTC m=+356.105713926" watchObservedRunningTime="2026-07-22 03:39:29.314920207 +0000 UTC m=+356.108408699" Jul 22 03:39:29.584829 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:29.584766 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:30.589996 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:30.589962 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:31.300849 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:31.300816 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:39:31.301917 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:39:31.301892 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf47mgtz" Jul 22 03:40:23.645496 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.645464 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r"] Jul 22 03:40:23.652550 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.652525 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.655671 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.655648 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 03:40:23.655773 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.655653 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 03:40:23.655773 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.655653 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-bbdd8\"" Jul 22 03:40:23.656323 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.656303 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r"] Jul 22 03:40:23.747782 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.747753 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z"] Jul 22 03:40:23.750551 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.750533 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:23.760116 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.760098 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z"] Jul 22 03:40:23.809123 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.809100 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wwxl\" (UniqueName: \"kubernetes.io/projected/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-kube-api-access-7wwxl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.809238 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.809135 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.809238 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.809167 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.910095 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910027 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wwxl\" (UniqueName: \"kubernetes.io/projected/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-kube-api-access-7wwxl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.910095 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910068 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.910267 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910177 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:23.910267 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910225 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.910412 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910271 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69hvj\" (UniqueName: \"kubernetes.io/projected/75de5bed-620f-49c6-8053-5c805123dd06-kube-api-access-69hvj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:23.910412 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910314 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:23.910517 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910471 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.910555 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.910526 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.919290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.919259 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wwxl\" (UniqueName: \"kubernetes.io/projected/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-kube-api-access-7wwxl\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:23.963398 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:23.963377 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:24.010940 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.010909 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:24.011777 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.011636 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:24.011777 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.011734 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69hvj\" (UniqueName: \"kubernetes.io/projected/75de5bed-620f-49c6-8053-5c805123dd06-kube-api-access-69hvj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:24.012026 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.011996 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:24.012138 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.012082 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:24.020866 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.020834 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69hvj\" (UniqueName: \"kubernetes.io/projected/75de5bed-620f-49c6-8053-5c805123dd06-kube-api-access-69hvj\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:24.049721 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.049669 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4"] Jul 22 03:40:24.054132 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.054113 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.059282 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.059250 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:24.061520 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.061162 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4"] Jul 22 03:40:24.100224 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.100201 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r"] Jul 22 03:40:24.102172 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:40:24.102144 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c456b6b_2ce6_45c8_8250_aa8d48efb99b.slice/crio-1145fa3723efb97c1c1bf7a444307f8e52b1ff5dca8b93a216867dc38442ae96 WatchSource:0}: Error finding container 1145fa3723efb97c1c1bf7a444307f8e52b1ff5dca8b93a216867dc38442ae96: Status 404 returned error can't find the container with id 1145fa3723efb97c1c1bf7a444307f8e52b1ff5dca8b93a216867dc38442ae96 Jul 22 03:40:24.200571 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.200534 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z"] Jul 22 03:40:24.200916 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:40:24.200895 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75de5bed_620f_49c6_8053_5c805123dd06.slice/crio-ee9b78f1c431524ded75e798c5a7eb74b0dd5925ed546699d0d5795ff470c64a WatchSource:0}: Error finding container ee9b78f1c431524ded75e798c5a7eb74b0dd5925ed546699d0d5795ff470c64a: Status 404 returned error can't find the container with id ee9b78f1c431524ded75e798c5a7eb74b0dd5925ed546699d0d5795ff470c64a Jul 22 03:40:24.213688 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.213640 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.213795 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.213693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rwch\" (UniqueName: \"kubernetes.io/projected/ac457ee9-3401-40e5-b31c-ed84f789875b-kube-api-access-8rwch\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.213795 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.213759 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.314822 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.314790 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.314974 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.314835 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rwch\" (UniqueName: \"kubernetes.io/projected/ac457ee9-3401-40e5-b31c-ed84f789875b-kube-api-access-8rwch\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.314974 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.314882 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.315139 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.315112 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.315252 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.315235 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.323595 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.323575 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rwch\" (UniqueName: \"kubernetes.io/projected/ac457ee9-3401-40e5-b31c-ed84f789875b-kube-api-access-8rwch\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.366053 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.366031 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:24.454312 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.454280 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5"] Jul 22 03:40:24.457090 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.457068 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.469312 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.469281 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5"] Jul 22 03:40:24.504032 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.504007 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4"] Jul 22 03:40:24.573740 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.573713 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" event={"ID":"ac457ee9-3401-40e5-b31c-ed84f789875b","Type":"ContainerStarted","Data":"112320f37be0834f4a3307828f696a4ed6ca3907da2f582028d312806d6507ac"} Jul 22 03:40:24.573870 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.573787 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" event={"ID":"ac457ee9-3401-40e5-b31c-ed84f789875b","Type":"ContainerStarted","Data":"b698fef368f4a54f545a5173e1bb9f3755bcac5bf35410c5d2e093c0042f038e"} Jul 22 03:40:24.575466 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.575437 2564 generic.go:356] "Generic (PLEG): container finished" podID="75de5bed-620f-49c6-8053-5c805123dd06" containerID="49928b14d3c6cf0c658e07a8a7480c2fe196615f9418458c032aaff2135b93fd" exitCode=0 Jul 22 03:40:24.575553 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.575516 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" event={"ID":"75de5bed-620f-49c6-8053-5c805123dd06","Type":"ContainerDied","Data":"49928b14d3c6cf0c658e07a8a7480c2fe196615f9418458c032aaff2135b93fd"} Jul 22 03:40:24.575609 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.575552 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" event={"ID":"75de5bed-620f-49c6-8053-5c805123dd06","Type":"ContainerStarted","Data":"ee9b78f1c431524ded75e798c5a7eb74b0dd5925ed546699d0d5795ff470c64a"} Jul 22 03:40:24.577483 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.577464 2564 generic.go:356] "Generic (PLEG): container finished" podID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerID="d5ea725c8403ed25bef3f6b3020cf981fad3cdaa2602a964c6ee30d7d43b6fcf" exitCode=0 Jul 22 03:40:24.577590 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.577504 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" event={"ID":"9c456b6b-2ce6-45c8-8250-aa8d48efb99b","Type":"ContainerDied","Data":"d5ea725c8403ed25bef3f6b3020cf981fad3cdaa2602a964c6ee30d7d43b6fcf"} Jul 22 03:40:24.577590 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.577538 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" event={"ID":"9c456b6b-2ce6-45c8-8250-aa8d48efb99b","Type":"ContainerStarted","Data":"1145fa3723efb97c1c1bf7a444307f8e52b1ff5dca8b93a216867dc38442ae96"} Jul 22 03:40:24.618799 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.618774 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.618909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.618825 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.618909 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.618848 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mdwh\" (UniqueName: \"kubernetes.io/projected/ffc675d9-96a6-487c-b309-b28cd00e46f6-kube-api-access-5mdwh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.719750 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.719682 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.719750 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.719743 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.720134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.719777 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mdwh\" (UniqueName: \"kubernetes.io/projected/ffc675d9-96a6-487c-b309-b28cd00e46f6-kube-api-access-5mdwh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.720134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.720040 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.720134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.720083 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.728082 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.728059 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mdwh\" (UniqueName: \"kubernetes.io/projected/ffc675d9-96a6-487c-b309-b28cd00e46f6-kube-api-access-5mdwh\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.769079 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.769055 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:24.894548 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:24.894521 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5"] Jul 22 03:40:24.896099 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:40:24.896077 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc675d9_96a6_487c_b309_b28cd00e46f6.slice/crio-f659b35eb69514597a6718ccf09b87017ae0776841ccea86d384a675970a92cc WatchSource:0}: Error finding container f659b35eb69514597a6718ccf09b87017ae0776841ccea86d384a675970a92cc: Status 404 returned error can't find the container with id f659b35eb69514597a6718ccf09b87017ae0776841ccea86d384a675970a92cc Jul 22 03:40:25.583170 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.583111 2564 generic.go:356] "Generic (PLEG): container finished" podID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerID="112320f37be0834f4a3307828f696a4ed6ca3907da2f582028d312806d6507ac" exitCode=0 Jul 22 03:40:25.583267 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.583200 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" event={"ID":"ac457ee9-3401-40e5-b31c-ed84f789875b","Type":"ContainerDied","Data":"112320f37be0834f4a3307828f696a4ed6ca3907da2f582028d312806d6507ac"} Jul 22 03:40:25.585179 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.585155 2564 generic.go:356] "Generic (PLEG): container finished" podID="75de5bed-620f-49c6-8053-5c805123dd06" containerID="a319d28b26786fa65236160ea5f0d756746c33c714db58d8b6cb6bee6234d653" exitCode=0 Jul 22 03:40:25.585284 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.585222 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" event={"ID":"75de5bed-620f-49c6-8053-5c805123dd06","Type":"ContainerDied","Data":"a319d28b26786fa65236160ea5f0d756746c33c714db58d8b6cb6bee6234d653"} Jul 22 03:40:25.587082 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.587061 2564 generic.go:356] "Generic (PLEG): container finished" podID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerID="a1c7d071a8823cb28230529c8e56e8f74c97d1c24ce5d2d134c46fab64550f17" exitCode=0 Jul 22 03:40:25.587163 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.587089 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" event={"ID":"9c456b6b-2ce6-45c8-8250-aa8d48efb99b","Type":"ContainerDied","Data":"a1c7d071a8823cb28230529c8e56e8f74c97d1c24ce5d2d134c46fab64550f17"} Jul 22 03:40:25.588718 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.588698 2564 generic.go:356] "Generic (PLEG): container finished" podID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerID="980ee2c11fe89b32eea1c19ed0ce9e7115b4f909d42ca8d3640464728156ea1b" exitCode=0 Jul 22 03:40:25.588822 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.588727 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" event={"ID":"ffc675d9-96a6-487c-b309-b28cd00e46f6","Type":"ContainerDied","Data":"980ee2c11fe89b32eea1c19ed0ce9e7115b4f909d42ca8d3640464728156ea1b"} Jul 22 03:40:25.588822 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:25.588758 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" event={"ID":"ffc675d9-96a6-487c-b309-b28cd00e46f6","Type":"ContainerStarted","Data":"f659b35eb69514597a6718ccf09b87017ae0776841ccea86d384a675970a92cc"} Jul 22 03:40:26.595608 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.595556 2564 generic.go:356] "Generic (PLEG): container finished" podID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerID="d370b507a49819424af4f04f7ab0dfcc0316f7b22832925f327d17dfcb98e3bc" exitCode=0 Jul 22 03:40:26.595907 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.595645 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" event={"ID":"ffc675d9-96a6-487c-b309-b28cd00e46f6","Type":"ContainerDied","Data":"d370b507a49819424af4f04f7ab0dfcc0316f7b22832925f327d17dfcb98e3bc"} Jul 22 03:40:26.597630 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.597610 2564 generic.go:356] "Generic (PLEG): container finished" podID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerID="dacc8325aed6f8a780e6aff38b2c48ff1c61ffddf315cc33b7ec410bea98625c" exitCode=0 Jul 22 03:40:26.597752 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.597693 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" event={"ID":"ac457ee9-3401-40e5-b31c-ed84f789875b","Type":"ContainerDied","Data":"dacc8325aed6f8a780e6aff38b2c48ff1c61ffddf315cc33b7ec410bea98625c"} Jul 22 03:40:26.599985 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.599957 2564 generic.go:356] "Generic (PLEG): container finished" podID="75de5bed-620f-49c6-8053-5c805123dd06" containerID="3a2120d86bceaa3c77279c641e7688c4992c455003cf3498fb7ec6073f218092" exitCode=0 Jul 22 03:40:26.600087 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.600062 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" event={"ID":"75de5bed-620f-49c6-8053-5c805123dd06","Type":"ContainerDied","Data":"3a2120d86bceaa3c77279c641e7688c4992c455003cf3498fb7ec6073f218092"} Jul 22 03:40:26.602033 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.602012 2564 generic.go:356] "Generic (PLEG): container finished" podID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerID="b3243960822e6b08c79b13dc148972658c43c542e772e4920e695c4f55603448" exitCode=0 Jul 22 03:40:26.602107 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:26.602037 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" event={"ID":"9c456b6b-2ce6-45c8-8250-aa8d48efb99b","Type":"ContainerDied","Data":"b3243960822e6b08c79b13dc148972658c43c542e772e4920e695c4f55603448"} Jul 22 03:40:27.608675 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.608645 2564 generic.go:356] "Generic (PLEG): container finished" podID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerID="0f55ee8d09e6b6bd086afb2973e302def07f97ffdd3b428a090f6a2781871893" exitCode=0 Jul 22 03:40:27.609079 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.608734 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" event={"ID":"ffc675d9-96a6-487c-b309-b28cd00e46f6","Type":"ContainerDied","Data":"0f55ee8d09e6b6bd086afb2973e302def07f97ffdd3b428a090f6a2781871893"} Jul 22 03:40:27.610707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.610679 2564 generic.go:356] "Generic (PLEG): container finished" podID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerID="2ae1da259fff2ef3545fdfcc6923eab6979a3667e7a18818114a4959b10daae8" exitCode=0 Jul 22 03:40:27.610809 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.610761 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" event={"ID":"ac457ee9-3401-40e5-b31c-ed84f789875b","Type":"ContainerDied","Data":"2ae1da259fff2ef3545fdfcc6923eab6979a3667e7a18818114a4959b10daae8"} Jul 22 03:40:27.745699 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.745676 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:27.780833 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.780806 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:27.842257 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.842228 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-bundle\") pod \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " Jul 22 03:40:27.842257 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.842260 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-util\") pod \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " Jul 22 03:40:27.842497 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.842302 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wwxl\" (UniqueName: \"kubernetes.io/projected/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-kube-api-access-7wwxl\") pod \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\" (UID: \"9c456b6b-2ce6-45c8-8250-aa8d48efb99b\") " Jul 22 03:40:27.842968 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.842934 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-bundle" (OuterVolumeSpecName: "bundle") pod "9c456b6b-2ce6-45c8-8250-aa8d48efb99b" (UID: "9c456b6b-2ce6-45c8-8250-aa8d48efb99b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:27.844672 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.844646 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-kube-api-access-7wwxl" (OuterVolumeSpecName: "kube-api-access-7wwxl") pod "9c456b6b-2ce6-45c8-8250-aa8d48efb99b" (UID: "9c456b6b-2ce6-45c8-8250-aa8d48efb99b"). InnerVolumeSpecName "kube-api-access-7wwxl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:40:27.848882 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.848859 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-util" (OuterVolumeSpecName: "util") pod "9c456b6b-2ce6-45c8-8250-aa8d48efb99b" (UID: "9c456b6b-2ce6-45c8-8250-aa8d48efb99b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:27.943681 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.943661 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-bundle\") pod \"75de5bed-620f-49c6-8053-5c805123dd06\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " Jul 22 03:40:27.943784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.943734 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-util\") pod \"75de5bed-620f-49c6-8053-5c805123dd06\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " Jul 22 03:40:27.943784 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.943761 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-69hvj\" (UniqueName: \"kubernetes.io/projected/75de5bed-620f-49c6-8053-5c805123dd06-kube-api-access-69hvj\") pod \"75de5bed-620f-49c6-8053-5c805123dd06\" (UID: \"75de5bed-620f-49c6-8053-5c805123dd06\") " Jul 22 03:40:27.943973 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.943959 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:27.944025 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.943978 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-util\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:27.944025 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.943991 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wwxl\" (UniqueName: \"kubernetes.io/projected/9c456b6b-2ce6-45c8-8250-aa8d48efb99b-kube-api-access-7wwxl\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:27.944230 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.944211 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-bundle" (OuterVolumeSpecName: "bundle") pod "75de5bed-620f-49c6-8053-5c805123dd06" (UID: "75de5bed-620f-49c6-8053-5c805123dd06"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:27.945764 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.945738 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75de5bed-620f-49c6-8053-5c805123dd06-kube-api-access-69hvj" (OuterVolumeSpecName: "kube-api-access-69hvj") pod "75de5bed-620f-49c6-8053-5c805123dd06" (UID: "75de5bed-620f-49c6-8053-5c805123dd06"). InnerVolumeSpecName "kube-api-access-69hvj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:40:27.948981 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:27.948951 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-util" (OuterVolumeSpecName: "util") pod "75de5bed-620f-49c6-8053-5c805123dd06" (UID: "75de5bed-620f-49c6-8053-5c805123dd06"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:28.044722 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.044701 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:28.044722 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.044719 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/75de5bed-620f-49c6-8053-5c805123dd06-util\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:28.044831 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.044728 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-69hvj\" (UniqueName: \"kubernetes.io/projected/75de5bed-620f-49c6-8053-5c805123dd06-kube-api-access-69hvj\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:28.617196 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.617175 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" Jul 22 03:40:28.617563 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.617173 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z" event={"ID":"75de5bed-620f-49c6-8053-5c805123dd06","Type":"ContainerDied","Data":"ee9b78f1c431524ded75e798c5a7eb74b0dd5925ed546699d0d5795ff470c64a"} Jul 22 03:40:28.617563 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.617356 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee9b78f1c431524ded75e798c5a7eb74b0dd5925ed546699d0d5795ff470c64a" Jul 22 03:40:28.619182 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.619162 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" Jul 22 03:40:28.619275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.619178 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r" event={"ID":"9c456b6b-2ce6-45c8-8250-aa8d48efb99b","Type":"ContainerDied","Data":"1145fa3723efb97c1c1bf7a444307f8e52b1ff5dca8b93a216867dc38442ae96"} Jul 22 03:40:28.619275 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.619218 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1145fa3723efb97c1c1bf7a444307f8e52b1ff5dca8b93a216867dc38442ae96" Jul 22 03:40:28.747796 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.747777 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:28.775406 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.775380 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:28.851603 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.851584 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-bundle\") pod \"ac457ee9-3401-40e5-b31c-ed84f789875b\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " Jul 22 03:40:28.851739 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.851636 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8rwch\" (UniqueName: \"kubernetes.io/projected/ac457ee9-3401-40e5-b31c-ed84f789875b-kube-api-access-8rwch\") pod \"ac457ee9-3401-40e5-b31c-ed84f789875b\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " Jul 22 03:40:28.851739 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.851664 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-util\") pod \"ac457ee9-3401-40e5-b31c-ed84f789875b\" (UID: \"ac457ee9-3401-40e5-b31c-ed84f789875b\") " Jul 22 03:40:28.852130 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.852104 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-bundle" (OuterVolumeSpecName: "bundle") pod "ac457ee9-3401-40e5-b31c-ed84f789875b" (UID: "ac457ee9-3401-40e5-b31c-ed84f789875b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:28.853763 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.853733 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac457ee9-3401-40e5-b31c-ed84f789875b-kube-api-access-8rwch" (OuterVolumeSpecName: "kube-api-access-8rwch") pod "ac457ee9-3401-40e5-b31c-ed84f789875b" (UID: "ac457ee9-3401-40e5-b31c-ed84f789875b"). InnerVolumeSpecName "kube-api-access-8rwch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:40:28.857715 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.857696 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-util" (OuterVolumeSpecName: "util") pod "ac457ee9-3401-40e5-b31c-ed84f789875b" (UID: "ac457ee9-3401-40e5-b31c-ed84f789875b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:28.952077 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.952055 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mdwh\" (UniqueName: \"kubernetes.io/projected/ffc675d9-96a6-487c-b309-b28cd00e46f6-kube-api-access-5mdwh\") pod \"ffc675d9-96a6-487c-b309-b28cd00e46f6\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " Jul 22 03:40:28.952142 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.952114 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-util\") pod \"ffc675d9-96a6-487c-b309-b28cd00e46f6\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " Jul 22 03:40:28.952186 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.952160 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-bundle\") pod \"ffc675d9-96a6-487c-b309-b28cd00e46f6\" (UID: \"ffc675d9-96a6-487c-b309-b28cd00e46f6\") " Jul 22 03:40:28.952317 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.952305 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:28.952402 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.952319 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8rwch\" (UniqueName: \"kubernetes.io/projected/ac457ee9-3401-40e5-b31c-ed84f789875b-kube-api-access-8rwch\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:28.952402 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.952329 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac457ee9-3401-40e5-b31c-ed84f789875b-util\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:28.952662 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.952641 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-bundle" (OuterVolumeSpecName: "bundle") pod "ffc675d9-96a6-487c-b309-b28cd00e46f6" (UID: "ffc675d9-96a6-487c-b309-b28cd00e46f6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:28.953990 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.953968 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffc675d9-96a6-487c-b309-b28cd00e46f6-kube-api-access-5mdwh" (OuterVolumeSpecName: "kube-api-access-5mdwh") pod "ffc675d9-96a6-487c-b309-b28cd00e46f6" (UID: "ffc675d9-96a6-487c-b309-b28cd00e46f6"). InnerVolumeSpecName "kube-api-access-5mdwh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:40:28.957191 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:28.957165 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-util" (OuterVolumeSpecName: "util") pod "ffc675d9-96a6-487c-b309-b28cd00e46f6" (UID: "ffc675d9-96a6-487c-b309-b28cd00e46f6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 03:40:29.053321 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.053291 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mdwh\" (UniqueName: \"kubernetes.io/projected/ffc675d9-96a6-487c-b309-b28cd00e46f6-kube-api-access-5mdwh\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:29.053321 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.053320 2564 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-util\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:29.053321 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.053328 2564 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ffc675d9-96a6-487c-b309-b28cd00e46f6-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:40:29.626119 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.626101 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" Jul 22 03:40:29.626478 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.626098 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5" event={"ID":"ffc675d9-96a6-487c-b309-b28cd00e46f6","Type":"ContainerDied","Data":"f659b35eb69514597a6718ccf09b87017ae0776841ccea86d384a675970a92cc"} Jul 22 03:40:29.626478 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.626208 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f659b35eb69514597a6718ccf09b87017ae0776841ccea86d384a675970a92cc" Jul 22 03:40:29.628007 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.627984 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" event={"ID":"ac457ee9-3401-40e5-b31c-ed84f789875b","Type":"ContainerDied","Data":"b698fef368f4a54f545a5173e1bb9f3755bcac5bf35410c5d2e093c0042f038e"} Jul 22 03:40:29.628007 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.628003 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4" Jul 22 03:40:29.628116 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:40:29.628024 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b698fef368f4a54f545a5173e1bb9f3755bcac5bf35410c5d2e093c0042f038e" Jul 22 03:41:27.678859 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.678829 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd"] Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679169 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679180 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679194 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75de5bed-620f-49c6-8053-5c805123dd06" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679201 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="75de5bed-620f-49c6-8053-5c805123dd06" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679209 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75de5bed-620f-49c6-8053-5c805123dd06" containerName="pull" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679213 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="75de5bed-620f-49c6-8053-5c805123dd06" containerName="pull" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679218 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerName="pull" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679223 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerName="pull" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679230 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerName="pull" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679235 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerName="pull" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679241 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679246 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679254 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679259 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerName="util" Jul 22 03:41:27.679269 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679271 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679279 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679288 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75de5bed-620f-49c6-8053-5c805123dd06" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679292 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="75de5bed-620f-49c6-8053-5c805123dd06" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679298 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerName="pull" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679303 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerName="pull" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679309 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679313 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679321 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679326 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679392 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="75de5bed-620f-49c6-8053-5c805123dd06" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679398 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c456b6b-2ce6-45c8-8250-aa8d48efb99b" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679406 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac457ee9-3401-40e5-b31c-ed84f789875b" containerName="extract" Jul 22 03:41:27.679707 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.679412 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="ffc675d9-96a6-487c-b309-b28cd00e46f6" containerName="extract" Jul 22 03:41:27.681598 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.681583 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.684673 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.684650 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-vg69p\"" Jul 22 03:41:27.695930 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.695912 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd"] Jul 22 03:41:27.762454 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762430 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762587 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762513 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762587 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762556 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762588 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762610 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762657 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab6787b-526f-438f-935b-8a400e7c4b68-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762694 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762689 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g78rd\" (UniqueName: \"kubernetes.io/projected/5ab6787b-526f-438f-935b-8a400e7c4b68-kube-api-access-g78rd\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762790 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.762934 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.762826 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863219 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863185 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab6787b-526f-438f-935b-8a400e7c4b68-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863229 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g78rd\" (UniqueName: \"kubernetes.io/projected/5ab6787b-526f-438f-935b-8a400e7c4b68-kube-api-access-g78rd\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863289 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863312 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863370 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863625 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863625 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863625 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863483 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863625 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863504 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863826 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863680 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-workload-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863826 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863730 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-data\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.863926 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863876 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-workload-certs\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.864000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863970 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5ab6787b-526f-438f-935b-8a400e7c4b68-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.864000 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.863994 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-credential-socket\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.866086 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.866058 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-podinfo\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.866186 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.866146 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-envoy\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.871738 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.871713 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5ab6787b-526f-438f-935b-8a400e7c4b68-istio-token\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.872394 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.872310 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g78rd\" (UniqueName: \"kubernetes.io/projected/5ab6787b-526f-438f-935b-8a400e7c4b68-kube-api-access-g78rd\") pod \"maas-default-gateway-openshift-default-56654dcf7-vrktd\" (UID: \"5ab6787b-526f-438f-935b-8a400e7c4b68\") " pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:27.991480 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:27.991454 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:28.123415 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.123386 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd"] Jul 22 03:41:28.124781 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:41:28.124760 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ab6787b_526f_438f_935b_8a400e7c4b68.slice/crio-5cbb2540f36201425caa91bdc9c8c7530db8f3da7dc5a9e07608e63414333136 WatchSource:0}: Error finding container 5cbb2540f36201425caa91bdc9c8c7530db8f3da7dc5a9e07608e63414333136: Status 404 returned error can't find the container with id 5cbb2540f36201425caa91bdc9c8c7530db8f3da7dc5a9e07608e63414333136 Jul 22 03:41:28.127081 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.127051 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:41:28.127160 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.127112 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:41:28.127160 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.127141 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:41:28.903449 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.903419 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" event={"ID":"5ab6787b-526f-438f-935b-8a400e7c4b68","Type":"ContainerStarted","Data":"d41a95f0f9103027e970413561c91f0e14470afee714a08c41598b31dc38ff12"} Jul 22 03:41:28.903782 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.903464 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" event={"ID":"5ab6787b-526f-438f-935b-8a400e7c4b68","Type":"ContainerStarted","Data":"5cbb2540f36201425caa91bdc9c8c7530db8f3da7dc5a9e07608e63414333136"} Jul 22 03:41:28.927539 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.927494 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" podStartSLOduration=1.927481399 podStartE2EDuration="1.927481399s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:41:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:41:28.925266991 +0000 UTC m=+475.718755482" watchObservedRunningTime="2026-07-22 03:41:28.927481399 +0000 UTC m=+475.720969890" Jul 22 03:41:28.992464 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:28.992440 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:29.996645 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:29.996620 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:30.918959 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:30.918925 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:30.919805 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:30.919783 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:41:40.396949 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.396917 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-688468586-b8t7m"] Jul 22 03:41:40.399397 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.399381 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.416573 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.416549 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688468586-b8t7m"] Jul 22 03:41:40.561507 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.561477 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-config\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.561649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.561512 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-oauth-serving-cert\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.561649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.561542 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-serving-cert\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.561649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.561591 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-trusted-ca-bundle\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.561649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.561644 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-service-ca\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.561831 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.561701 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-oauth-config\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.561831 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.561767 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkr47\" (UniqueName: \"kubernetes.io/projected/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-kube-api-access-rkr47\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.662578 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.662508 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-oauth-serving-cert\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.662578 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.662544 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-serving-cert\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.662578 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.662563 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-trusted-ca-bundle\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.662818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.662592 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-service-ca\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.662818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.662629 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-oauth-config\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.662818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.662677 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkr47\" (UniqueName: \"kubernetes.io/projected/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-kube-api-access-rkr47\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.662818 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.662731 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-config\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.663613 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.663536 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-service-ca\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.663613 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.663654 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-config\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.663863 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.663678 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-oauth-serving-cert\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.663863 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.663738 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-trusted-ca-bundle\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.665735 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.665699 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-serving-cert\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.665860 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.665751 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-console-oauth-config\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.672014 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.671986 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkr47\" (UniqueName: \"kubernetes.io/projected/0bda44b0-94ae-4dd3-8e2a-e71517e164c1-kube-api-access-rkr47\") pod \"console-688468586-b8t7m\" (UID: \"0bda44b0-94ae-4dd3-8e2a-e71517e164c1\") " pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.709157 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.709135 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:40.841101 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.841077 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-688468586-b8t7m"] Jul 22 03:41:40.842136 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:41:40.842106 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bda44b0_94ae_4dd3_8e2a_e71517e164c1.slice/crio-b371002a369ea060b49d528a5a58e0461f68e5a7901aa50838afa29ffd51a264 WatchSource:0}: Error finding container b371002a369ea060b49d528a5a58e0461f68e5a7901aa50838afa29ffd51a264: Status 404 returned error can't find the container with id b371002a369ea060b49d528a5a58e0461f68e5a7901aa50838afa29ffd51a264 Jul 22 03:41:40.966998 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.966930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688468586-b8t7m" event={"ID":"0bda44b0-94ae-4dd3-8e2a-e71517e164c1","Type":"ContainerStarted","Data":"ddcf06c11044afafb2e1e4939e219ab8f13a693d29aaf35a09bc1546e899e7e3"} Jul 22 03:41:40.966998 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.966978 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-688468586-b8t7m" event={"ID":"0bda44b0-94ae-4dd3-8e2a-e71517e164c1","Type":"ContainerStarted","Data":"b371002a369ea060b49d528a5a58e0461f68e5a7901aa50838afa29ffd51a264"} Jul 22 03:41:40.985950 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:40.985906 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-688468586-b8t7m" podStartSLOduration=0.985890867 podStartE2EDuration="985.890867ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:41:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:41:40.984547567 +0000 UTC m=+487.778036061" watchObservedRunningTime="2026-07-22 03:41:40.985890867 +0000 UTC m=+487.779379359" Jul 22 03:41:50.709495 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:50.709462 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:50.709978 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:50.709516 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:50.714480 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:50.714457 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:51.024434 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:51.024409 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-688468586-b8t7m" Jul 22 03:41:51.073942 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:41:51.073911 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f8b8b4f7-rhzk9"] Jul 22 03:42:10.421465 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.421430 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-65458f5dc4-2l6fk"] Jul 22 03:42:10.423918 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.423897 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.427064 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.427044 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 03:42:10.427394 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.427378 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 03:42:10.427478 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.427407 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fkk4q\"" Jul 22 03:42:10.441417 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.441397 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65458f5dc4-2l6fk"] Jul 22 03:42:10.508507 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.508470 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fhbn\" (UniqueName: \"kubernetes.io/projected/cca001a7-9035-4c32-bac3-83070fa83c74-kube-api-access-9fhbn\") pod \"maas-controller-65458f5dc4-2l6fk\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.508649 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.508603 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cca001a7-9035-4c32-bac3-83070fa83c74-webhook-cert\") pod \"maas-controller-65458f5dc4-2l6fk\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.609480 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.609451 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cca001a7-9035-4c32-bac3-83070fa83c74-webhook-cert\") pod \"maas-controller-65458f5dc4-2l6fk\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.609604 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.609503 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9fhbn\" (UniqueName: \"kubernetes.io/projected/cca001a7-9035-4c32-bac3-83070fa83c74-kube-api-access-9fhbn\") pod \"maas-controller-65458f5dc4-2l6fk\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.611866 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.611846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cca001a7-9035-4c32-bac3-83070fa83c74-webhook-cert\") pod \"maas-controller-65458f5dc4-2l6fk\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.623966 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.623941 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fhbn\" (UniqueName: \"kubernetes.io/projected/cca001a7-9035-4c32-bac3-83070fa83c74-kube-api-access-9fhbn\") pod \"maas-controller-65458f5dc4-2l6fk\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.734322 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.734268 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:10.866250 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:10.866220 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-65458f5dc4-2l6fk"] Jul 22 03:42:10.867509 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:42:10.867486 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcca001a7_9035_4c32_bac3_83070fa83c74.slice/crio-0e85d2afa1a05aa0499cb77681350133b7a2bc45a3d8e71570a556a103083d54 WatchSource:0}: Error finding container 0e85d2afa1a05aa0499cb77681350133b7a2bc45a3d8e71570a556a103083d54: Status 404 returned error can't find the container with id 0e85d2afa1a05aa0499cb77681350133b7a2bc45a3d8e71570a556a103083d54 Jul 22 03:42:11.121611 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:11.121578 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" event={"ID":"cca001a7-9035-4c32-bac3-83070fa83c74","Type":"ContainerStarted","Data":"0e85d2afa1a05aa0499cb77681350133b7a2bc45a3d8e71570a556a103083d54"} Jul 22 03:42:13.135802 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:13.135767 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" event={"ID":"cca001a7-9035-4c32-bac3-83070fa83c74","Type":"ContainerStarted","Data":"1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173"} Jul 22 03:42:13.136107 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:13.135830 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:13.155261 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:13.155214 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" podStartSLOduration=0.959091876 podStartE2EDuration="3.155200148s" totalImagesPullingTime="2.196108272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:42:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:42:13.15399719 +0000 UTC m=+519.947485686" watchObservedRunningTime="2026-07-22 03:42:13.155200148 +0000 UTC m=+519.948688643" Jul 22 03:42:16.100930 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.100874 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64f8b8b4f7-rhzk9" podUID="570bd4fa-1209-4755-a7f4-a60b048ad9a3" containerName="console" containerID="cri-o://b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e" gracePeriod=15 Jul 22 03:42:16.350364 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.350327 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:42:16.459526 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459440 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-oauth-serving-cert\") pod \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " Jul 22 03:42:16.459526 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459475 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-config\") pod \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " Jul 22 03:42:16.459526 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459512 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g2ktw\" (UniqueName: \"kubernetes.io/projected/570bd4fa-1209-4755-a7f4-a60b048ad9a3-kube-api-access-g2ktw\") pod \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " Jul 22 03:42:16.459787 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459534 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-service-ca\") pod \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " Jul 22 03:42:16.459787 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459597 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-serving-cert\") pod \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " Jul 22 03:42:16.459787 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459623 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-oauth-config\") pod \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " Jul 22 03:42:16.459787 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459656 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-trusted-ca-bundle\") pod \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\" (UID: \"570bd4fa-1209-4755-a7f4-a60b048ad9a3\") " Jul 22 03:42:16.459989 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459871 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-config" (OuterVolumeSpecName: "console-config") pod "570bd4fa-1209-4755-a7f4-a60b048ad9a3" (UID: "570bd4fa-1209-4755-a7f4-a60b048ad9a3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:42:16.460052 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.459913 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "570bd4fa-1209-4755-a7f4-a60b048ad9a3" (UID: "570bd4fa-1209-4755-a7f4-a60b048ad9a3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:42:16.460159 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.460128 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "570bd4fa-1209-4755-a7f4-a60b048ad9a3" (UID: "570bd4fa-1209-4755-a7f4-a60b048ad9a3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:42:16.460280 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.460258 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-service-ca" (OuterVolumeSpecName: "service-ca") pod "570bd4fa-1209-4755-a7f4-a60b048ad9a3" (UID: "570bd4fa-1209-4755-a7f4-a60b048ad9a3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:42:16.461879 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.461856 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "570bd4fa-1209-4755-a7f4-a60b048ad9a3" (UID: "570bd4fa-1209-4755-a7f4-a60b048ad9a3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:42:16.461956 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.461879 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "570bd4fa-1209-4755-a7f4-a60b048ad9a3" (UID: "570bd4fa-1209-4755-a7f4-a60b048ad9a3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:42:16.461956 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.461938 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/570bd4fa-1209-4755-a7f4-a60b048ad9a3-kube-api-access-g2ktw" (OuterVolumeSpecName: "kube-api-access-g2ktw") pod "570bd4fa-1209-4755-a7f4-a60b048ad9a3" (UID: "570bd4fa-1209-4755-a7f4-a60b048ad9a3"). InnerVolumeSpecName "kube-api-access-g2ktw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:42:16.560886 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.560853 2564 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:16.560886 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.560880 2564 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:16.560886 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.560890 2564 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-oauth-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:16.561102 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.560898 2564 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-trusted-ca-bundle\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:16.561102 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.560907 2564 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-oauth-serving-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:16.561102 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.560915 2564 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/570bd4fa-1209-4755-a7f4-a60b048ad9a3-console-config\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:16.561102 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:16.560922 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g2ktw\" (UniqueName: \"kubernetes.io/projected/570bd4fa-1209-4755-a7f4-a60b048ad9a3-kube-api-access-g2ktw\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:17.157471 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.157430 2564 generic.go:356] "Generic (PLEG): container finished" podID="570bd4fa-1209-4755-a7f4-a60b048ad9a3" containerID="b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e" exitCode=0 Jul 22 03:42:17.157881 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.157479 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64f8b8b4f7-rhzk9" Jul 22 03:42:17.157881 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.157518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f8b8b4f7-rhzk9" event={"ID":"570bd4fa-1209-4755-a7f4-a60b048ad9a3","Type":"ContainerDied","Data":"b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e"} Jul 22 03:42:17.157881 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.157571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64f8b8b4f7-rhzk9" event={"ID":"570bd4fa-1209-4755-a7f4-a60b048ad9a3","Type":"ContainerDied","Data":"33c1097426b712bd5d9ec4407b665f9d0b426f0547474269490e295c32a114f8"} Jul 22 03:42:17.157881 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.157592 2564 scope.go:117] "RemoveContainer" containerID="b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e" Jul 22 03:42:17.167434 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.167417 2564 scope.go:117] "RemoveContainer" containerID="b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e" Jul 22 03:42:17.167660 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:42:17.167640 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e\": container with ID starting with b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e not found: ID does not exist" containerID="b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e" Jul 22 03:42:17.167714 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.167669 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e"} err="failed to get container status \"b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e\": rpc error: code = NotFound desc = could not find container \"b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e\": container with ID starting with b1b1cf9f594a4e3be65202d8b1c2004149e7f9e62f89971a5ebad96516bc670e not found: ID does not exist" Jul 22 03:42:17.181079 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.181055 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64f8b8b4f7-rhzk9"] Jul 22 03:42:17.184983 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.184964 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64f8b8b4f7-rhzk9"] Jul 22 03:42:17.746061 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:17.746026 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="570bd4fa-1209-4755-a7f4-a60b048ad9a3" path="/var/lib/kubelet/pods/570bd4fa-1209-4755-a7f4-a60b048ad9a3/volumes" Jul 22 03:42:24.143714 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:24.143632 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:24.754621 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:24.754582 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65458f5dc4-2l6fk"] Jul 22 03:42:24.754816 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:24.754776 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" podUID="cca001a7-9035-4c32-bac3-83070fa83c74" containerName="manager" containerID="cri-o://1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173" gracePeriod=30 Jul 22 03:42:25.004580 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.004557 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:25.132525 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.132459 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9fhbn\" (UniqueName: \"kubernetes.io/projected/cca001a7-9035-4c32-bac3-83070fa83c74-kube-api-access-9fhbn\") pod \"cca001a7-9035-4c32-bac3-83070fa83c74\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " Jul 22 03:42:25.132525 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.132505 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cca001a7-9035-4c32-bac3-83070fa83c74-webhook-cert\") pod \"cca001a7-9035-4c32-bac3-83070fa83c74\" (UID: \"cca001a7-9035-4c32-bac3-83070fa83c74\") " Jul 22 03:42:25.134749 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.134723 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cca001a7-9035-4c32-bac3-83070fa83c74-kube-api-access-9fhbn" (OuterVolumeSpecName: "kube-api-access-9fhbn") pod "cca001a7-9035-4c32-bac3-83070fa83c74" (UID: "cca001a7-9035-4c32-bac3-83070fa83c74"). InnerVolumeSpecName "kube-api-access-9fhbn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:42:25.134812 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.134780 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cca001a7-9035-4c32-bac3-83070fa83c74-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "cca001a7-9035-4c32-bac3-83070fa83c74" (UID: "cca001a7-9035-4c32-bac3-83070fa83c74"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:42:25.198328 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.198299 2564 generic.go:356] "Generic (PLEG): container finished" podID="cca001a7-9035-4c32-bac3-83070fa83c74" containerID="1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173" exitCode=0 Jul 22 03:42:25.198652 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.198371 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" Jul 22 03:42:25.198652 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.198382 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" event={"ID":"cca001a7-9035-4c32-bac3-83070fa83c74","Type":"ContainerDied","Data":"1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173"} Jul 22 03:42:25.198652 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.198437 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-65458f5dc4-2l6fk" event={"ID":"cca001a7-9035-4c32-bac3-83070fa83c74","Type":"ContainerDied","Data":"0e85d2afa1a05aa0499cb77681350133b7a2bc45a3d8e71570a556a103083d54"} Jul 22 03:42:25.198652 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.198460 2564 scope.go:117] "RemoveContainer" containerID="1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173" Jul 22 03:42:25.207721 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.207702 2564 scope.go:117] "RemoveContainer" containerID="1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173" Jul 22 03:42:25.207974 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:42:25.207954 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173\": container with ID starting with 1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173 not found: ID does not exist" containerID="1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173" Jul 22 03:42:25.208017 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.207983 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173"} err="failed to get container status \"1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173\": rpc error: code = NotFound desc = could not find container \"1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173\": container with ID starting with 1c6dfcf072b2decf1691167d65e122ec7c0922c5f1d471555741db8ddeeab173 not found: ID does not exist" Jul 22 03:42:25.224533 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.224504 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-65458f5dc4-2l6fk"] Jul 22 03:42:25.229680 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.229662 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-65458f5dc4-2l6fk"] Jul 22 03:42:25.233745 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.233720 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9fhbn\" (UniqueName: \"kubernetes.io/projected/cca001a7-9035-4c32-bac3-83070fa83c74-kube-api-access-9fhbn\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:25.233815 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.233749 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cca001a7-9035-4c32-bac3-83070fa83c74-webhook-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:42:25.744671 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:25.744645 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cca001a7-9035-4c32-bac3-83070fa83c74" path="/var/lib/kubelet/pods/cca001a7-9035-4c32-bac3-83070fa83c74/volumes" Jul 22 03:42:46.273358 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.273309 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l4skc"] Jul 22 03:42:46.273905 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.273820 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cca001a7-9035-4c32-bac3-83070fa83c74" containerName="manager" Jul 22 03:42:46.273905 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.273839 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="cca001a7-9035-4c32-bac3-83070fa83c74" containerName="manager" Jul 22 03:42:46.273905 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.273865 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="570bd4fa-1209-4755-a7f4-a60b048ad9a3" containerName="console" Jul 22 03:42:46.273905 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.273873 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="570bd4fa-1209-4755-a7f4-a60b048ad9a3" containerName="console" Jul 22 03:42:46.274116 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.273955 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="570bd4fa-1209-4755-a7f4-a60b048ad9a3" containerName="console" Jul 22 03:42:46.274116 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.273969 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="cca001a7-9035-4c32-bac3-83070fa83c74" containerName="manager" Jul 22 03:42:46.279068 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.279047 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.282325 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.282307 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 22 03:42:46.282451 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.282428 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-tkpz6\"" Jul 22 03:42:46.294609 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.294587 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l4skc"] Jul 22 03:42:46.400724 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.400693 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfbd7\" (UniqueName: \"kubernetes.io/projected/95b1b413-599a-474c-8987-a7786d3c6817-kube-api-access-wfbd7\") pod \"payload-processing-5bf47cd988-l4skc\" (UID: \"95b1b413-599a-474c-8987-a7786d3c6817\") " pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.400865 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.400736 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b1b413-599a-474c-8987-a7786d3c6817-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l4skc\" (UID: \"95b1b413-599a-474c-8987-a7786d3c6817\") " pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.501166 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.501137 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfbd7\" (UniqueName: \"kubernetes.io/projected/95b1b413-599a-474c-8987-a7786d3c6817-kube-api-access-wfbd7\") pod \"payload-processing-5bf47cd988-l4skc\" (UID: \"95b1b413-599a-474c-8987-a7786d3c6817\") " pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.501298 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.501180 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b1b413-599a-474c-8987-a7786d3c6817-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l4skc\" (UID: \"95b1b413-599a-474c-8987-a7786d3c6817\") " pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.501875 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.501856 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/95b1b413-599a-474c-8987-a7786d3c6817-plugins-config-volume\") pod \"payload-processing-5bf47cd988-l4skc\" (UID: \"95b1b413-599a-474c-8987-a7786d3c6817\") " pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.511209 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.511178 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfbd7\" (UniqueName: \"kubernetes.io/projected/95b1b413-599a-474c-8987-a7786d3c6817-kube-api-access-wfbd7\") pod \"payload-processing-5bf47cd988-l4skc\" (UID: \"95b1b413-599a-474c-8987-a7786d3c6817\") " pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.589299 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.589245 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:46.723021 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:46.722991 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-l4skc"] Jul 22 03:42:46.724775 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:42:46.724751 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95b1b413_599a_474c_8987_a7786d3c6817.slice/crio-6ddca830c6359eb125ece8538313e8365f185965f28e54eaea34e9aa8aa6b67a WatchSource:0}: Error finding container 6ddca830c6359eb125ece8538313e8365f185965f28e54eaea34e9aa8aa6b67a: Status 404 returned error can't find the container with id 6ddca830c6359eb125ece8538313e8365f185965f28e54eaea34e9aa8aa6b67a Jul 22 03:42:47.330089 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:47.330053 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" event={"ID":"95b1b413-599a-474c-8987-a7786d3c6817","Type":"ContainerStarted","Data":"6ddca830c6359eb125ece8538313e8365f185965f28e54eaea34e9aa8aa6b67a"} Jul 22 03:42:49.342416 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:49.342379 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" event={"ID":"95b1b413-599a-474c-8987-a7786d3c6817","Type":"ContainerStarted","Data":"4a578f55c8199dc7b5fac264da06360018156630cd35b22f3eee46c2e2b77619"} Jul 22 03:42:49.342781 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:49.342445 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:42:49.360586 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:49.360538 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" podStartSLOduration=1.519471976 podStartE2EDuration="3.360524618s" totalImagesPullingTime="1.841052642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:42:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:42:49.3594689 +0000 UTC m=+556.152957404" watchObservedRunningTime="2026-07-22 03:42:49.360524618 +0000 UTC m=+556.154013109" Jul 22 03:42:54.374745 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:54.374711 2564 generic.go:356] "Generic (PLEG): container finished" podID="5ab6787b-526f-438f-935b-8a400e7c4b68" containerID="d41a95f0f9103027e970413561c91f0e14470afee714a08c41598b31dc38ff12" exitCode=0 Jul 22 03:42:54.375199 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:54.374756 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" event={"ID":"5ab6787b-526f-438f-935b-8a400e7c4b68","Type":"ContainerDied","Data":"d41a95f0f9103027e970413561c91f0e14470afee714a08c41598b31dc38ff12"} Jul 22 03:42:54.375199 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:54.375122 2564 scope.go:117] "RemoveContainer" containerID="d41a95f0f9103027e970413561c91f0e14470afee714a08c41598b31dc38ff12" Jul 22 03:42:54.376138 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:54.376106 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:42:54.376216 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:54.376196 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:42:54.376264 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:54.376250 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 03:42:55.387551 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:55.387509 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" event={"ID":"5ab6787b-526f-438f-935b-8a400e7c4b68","Type":"ContainerStarted","Data":"474fcc769404e216ee926385abe8dd53e72c668f9304ecd229f6f18935bf0780"} Jul 22 03:42:55.992455 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:55.992419 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:42:55.993188 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:55.992728 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vrktd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 22 03:42:55.993410 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:55.993382 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" podUID="5ab6787b-526f-438f-935b-8a400e7c4b68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 22 03:42:56.992749 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:56.992711 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vrktd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 22 03:42:56.993354 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:56.993303 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" podUID="5ab6787b-526f-438f-935b-8a400e7c4b68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 22 03:42:57.991994 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:57.991957 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vrktd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 22 03:42:57.992276 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:57.992247 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" podUID="5ab6787b-526f-438f-935b-8a400e7c4b68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 22 03:42:58.992798 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:58.992760 2564 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-56654dcf7-vrktd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Jul 22 03:42:58.993443 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:58.993407 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" podUID="5ab6787b-526f-438f-935b-8a400e7c4b68" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Jul 22 03:42:59.996590 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:59.996558 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:42:59.996977 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:59.996943 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:42:59.997653 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:42:59.997630 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-vrktd" Jul 22 03:43:00.354290 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:00.354189 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-l4skc" Jul 22 03:43:03.063609 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.063578 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4"] Jul 22 03:43:03.065995 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.065980 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.068857 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.068834 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vmmxq\"" Jul 22 03:43:03.068968 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.068895 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 03:43:03.070128 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.070110 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 22 03:43:03.070244 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.070165 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 03:43:03.082046 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.082019 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4"] Jul 22 03:43:03.144264 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.144238 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.144400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.144271 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.144400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.144315 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.144400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.144348 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.144400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.144375 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.144400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.144389 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j56fg\" (UniqueName: \"kubernetes.io/projected/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-kube-api-access-j56fg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.244912 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.244886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.244917 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.244947 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.244962 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j56fg\" (UniqueName: \"kubernetes.io/projected/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-kube-api-access-j56fg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.245017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245044 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.245042 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245391 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.245373 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245466 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.245412 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.245504 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.245469 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.247471 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.247450 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.247755 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.247736 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.253586 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.253566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j56fg\" (UniqueName: \"kubernetes.io/projected/b0cb81ef-1f92-48c2-9fda-a532a33e70ad-kube-api-access-j56fg\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4\" (UID: \"b0cb81ef-1f92-48c2-9fda-a532a33e70ad\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.376400 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.376306 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:03.507712 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:03.507687 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4"] Jul 22 03:43:03.509120 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:43:03.509092 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0cb81ef_1f92_48c2_9fda_a532a33e70ad.slice/crio-4053c393a95ea98f073776335ae43a87fb66234c20788728a1dee340823ac615 WatchSource:0}: Error finding container 4053c393a95ea98f073776335ae43a87fb66234c20788728a1dee340823ac615: Status 404 returned error can't find the container with id 4053c393a95ea98f073776335ae43a87fb66234c20788728a1dee340823ac615 Jul 22 03:43:04.444954 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:04.444925 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" event={"ID":"b0cb81ef-1f92-48c2-9fda-a532a33e70ad","Type":"ContainerStarted","Data":"4053c393a95ea98f073776335ae43a87fb66234c20788728a1dee340823ac615"} Jul 22 03:43:05.452470 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:05.452428 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" event={"ID":"b0cb81ef-1f92-48c2-9fda-a532a33e70ad","Type":"ContainerStarted","Data":"e9891a8374b1a4b5d3407c1eafc3b4339d8dedd8d131dfcb0c489a379c708cc2"} Jul 22 03:43:05.471311 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:05.471261 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" podStartSLOduration=1.493988011 podStartE2EDuration="2.471244884s" totalImagesPullingTime="977.256873ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:43:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:43:05.470152644 +0000 UTC m=+572.263641138" watchObservedRunningTime="2026-07-22 03:43:05.471244884 +0000 UTC m=+572.264733377" Jul 22 03:43:13.376444 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:13.376407 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:13.376815 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:13.376468 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:13.388943 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:13.388919 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:43:13.506461 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:43:13.506434 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4" Jul 22 03:45:00.247364 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:00.247298 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2"] Jul 22 03:45:00.250097 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:00.250067 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" Jul 22 03:45:00.257659 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:00.257318 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\"" Jul 22 03:45:00.260611 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:00.260586 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" Jul 22 03:45:00.285552 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:00.285523 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2"] Jul 22 03:45:00.411995 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:00.411974 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2"] Jul 22 03:45:00.414092 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:45:00.414060 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb7b71fe_da49_4a3a_bab5_be83a122a20a.slice/crio-0779f99963c23929a2f10970524b52e52a777bff9c154d531c0905e55d58501c WatchSource:0}: Error finding container 0779f99963c23929a2f10970524b52e52a777bff9c154d531c0905e55d58501c: Status 404 returned error can't find the container with id 0779f99963c23929a2f10970524b52e52a777bff9c154d531c0905e55d58501c Jul 22 03:45:00.416151 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:00.416134 2564 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 03:45:01.066388 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:01.066329 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" event={"ID":"db7b71fe-da49-4a3a-bab5-be83a122a20a","Type":"ContainerStarted","Data":"0779f99963c23929a2f10970524b52e52a777bff9c154d531c0905e55d58501c"} Jul 22 03:45:03.083129 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:03.083092 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" event={"ID":"db7b71fe-da49-4a3a-bab5-be83a122a20a","Type":"ContainerStarted","Data":"b2e96f25479335f9fa11de517403291a73672a7d21956157329a1a0fe29774bf"} Jul 22 03:45:03.123438 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:03.123394 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" podStartSLOduration=1.001942956 podStartE2EDuration="3.123379101s" totalImagesPullingTime="2.121436145s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:45:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:45:03.122200319 +0000 UTC m=+689.915688810" watchObservedRunningTime="2026-07-22 03:45:03.123379101 +0000 UTC m=+689.916867593" Jul 22 03:45:04.088742 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:04.088709 2564 generic.go:356] "Generic (PLEG): container finished" podID="db7b71fe-da49-4a3a-bab5-be83a122a20a" containerID="b2e96f25479335f9fa11de517403291a73672a7d21956157329a1a0fe29774bf" exitCode=0 Jul 22 03:45:04.089167 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:04.088791 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" event={"ID":"db7b71fe-da49-4a3a-bab5-be83a122a20a","Type":"ContainerDied","Data":"b2e96f25479335f9fa11de517403291a73672a7d21956157329a1a0fe29774bf"} Jul 22 03:45:05.232971 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:05.232943 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" Jul 22 03:45:06.101393 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:06.101313 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" Jul 22 03:45:06.101393 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:06.101374 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744865-twrw2" event={"ID":"db7b71fe-da49-4a3a-bab5-be83a122a20a","Type":"ContainerDied","Data":"0779f99963c23929a2f10970524b52e52a777bff9c154d531c0905e55d58501c"} Jul 22 03:45:06.101610 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:45:06.101428 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0779f99963c23929a2f10970524b52e52a777bff9c154d531c0905e55d58501c" Jul 22 03:46:46.593372 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.593321 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74d8745f54-fv6nt"] Jul 22 03:46:46.593821 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.593697 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db7b71fe-da49-4a3a-bab5-be83a122a20a" containerName="cleanup" Jul 22 03:46:46.593821 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.593709 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="db7b71fe-da49-4a3a-bab5-be83a122a20a" containerName="cleanup" Jul 22 03:46:46.593821 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.593783 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="db7b71fe-da49-4a3a-bab5-be83a122a20a" containerName="cleanup" Jul 22 03:46:46.596960 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.596936 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:46.601317 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.601294 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fkk4q\"" Jul 22 03:46:46.601458 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.601321 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 03:46:46.601458 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.601300 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 03:46:46.615701 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.615663 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d8745f54-fv6nt"] Jul 22 03:46:46.760786 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.760750 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rr4h\" (UniqueName: \"kubernetes.io/projected/0b146bba-088f-4f7f-b687-20fda1fa2e4f-kube-api-access-9rr4h\") pod \"maas-controller-74d8745f54-fv6nt\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:46.760914 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.760817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b146bba-088f-4f7f-b687-20fda1fa2e4f-webhook-cert\") pod \"maas-controller-74d8745f54-fv6nt\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:46.862145 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.862067 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b146bba-088f-4f7f-b687-20fda1fa2e4f-webhook-cert\") pod \"maas-controller-74d8745f54-fv6nt\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:46.862145 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.862134 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rr4h\" (UniqueName: \"kubernetes.io/projected/0b146bba-088f-4f7f-b687-20fda1fa2e4f-kube-api-access-9rr4h\") pod \"maas-controller-74d8745f54-fv6nt\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:46.864594 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.864575 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b146bba-088f-4f7f-b687-20fda1fa2e4f-webhook-cert\") pod \"maas-controller-74d8745f54-fv6nt\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:46.871903 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.871880 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rr4h\" (UniqueName: \"kubernetes.io/projected/0b146bba-088f-4f7f-b687-20fda1fa2e4f-kube-api-access-9rr4h\") pod \"maas-controller-74d8745f54-fv6nt\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:46.908071 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:46.908046 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:47.045495 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:47.045466 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d8745f54-fv6nt"] Jul 22 03:46:47.047165 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:46:47.047140 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b146bba_088f_4f7f_b687_20fda1fa2e4f.slice/crio-dc1b4e97f701ab4fdababdd11a071c55e0f0fbf5e942211a717fa5a88745e44b WatchSource:0}: Error finding container dc1b4e97f701ab4fdababdd11a071c55e0f0fbf5e942211a717fa5a88745e44b: Status 404 returned error can't find the container with id dc1b4e97f701ab4fdababdd11a071c55e0f0fbf5e942211a717fa5a88745e44b Jul 22 03:46:47.627208 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:47.627173 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d8745f54-fv6nt" event={"ID":"0b146bba-088f-4f7f-b687-20fda1fa2e4f","Type":"ContainerStarted","Data":"9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452"} Jul 22 03:46:47.627654 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:47.627222 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d8745f54-fv6nt" event={"ID":"0b146bba-088f-4f7f-b687-20fda1fa2e4f","Type":"ContainerStarted","Data":"dc1b4e97f701ab4fdababdd11a071c55e0f0fbf5e942211a717fa5a88745e44b"} Jul 22 03:46:47.627654 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:47.627308 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:46:47.646688 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:47.646647 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74d8745f54-fv6nt" podStartSLOduration=1.2478415680000001 podStartE2EDuration="1.646633114s" totalImagesPullingTime="398.791546ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:46:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:46:47.644739881 +0000 UTC m=+794.438228386" watchObservedRunningTime="2026-07-22 03:46:47.646633114 +0000 UTC m=+794.440121606" Jul 22 03:46:58.635081 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:46:58.635003 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:47:03.297473 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.297439 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d8745f54-fv6nt"] Jul 22 03:47:03.297973 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.297660 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-74d8745f54-fv6nt" podUID="0b146bba-088f-4f7f-b687-20fda1fa2e4f" containerName="manager" containerID="cri-o://9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452" gracePeriod=30 Jul 22 03:47:03.547833 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.547770 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:47:03.606360 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.606309 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b146bba-088f-4f7f-b687-20fda1fa2e4f-webhook-cert\") pod \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " Jul 22 03:47:03.606521 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.606411 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rr4h\" (UniqueName: \"kubernetes.io/projected/0b146bba-088f-4f7f-b687-20fda1fa2e4f-kube-api-access-9rr4h\") pod \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\" (UID: \"0b146bba-088f-4f7f-b687-20fda1fa2e4f\") " Jul 22 03:47:03.608491 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.608465 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b146bba-088f-4f7f-b687-20fda1fa2e4f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "0b146bba-088f-4f7f-b687-20fda1fa2e4f" (UID: "0b146bba-088f-4f7f-b687-20fda1fa2e4f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:47:03.608581 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.608491 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b146bba-088f-4f7f-b687-20fda1fa2e4f-kube-api-access-9rr4h" (OuterVolumeSpecName: "kube-api-access-9rr4h") pod "0b146bba-088f-4f7f-b687-20fda1fa2e4f" (UID: "0b146bba-088f-4f7f-b687-20fda1fa2e4f"). InnerVolumeSpecName "kube-api-access-9rr4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:47:03.707430 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.707400 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rr4h\" (UniqueName: \"kubernetes.io/projected/0b146bba-088f-4f7f-b687-20fda1fa2e4f-kube-api-access-9rr4h\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:47:03.707430 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.707423 2564 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0b146bba-088f-4f7f-b687-20fda1fa2e4f-webhook-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 03:47:03.717675 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.717652 2564 generic.go:356] "Generic (PLEG): container finished" podID="0b146bba-088f-4f7f-b687-20fda1fa2e4f" containerID="9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452" exitCode=0 Jul 22 03:47:03.717795 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.717696 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d8745f54-fv6nt" Jul 22 03:47:03.717795 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.717754 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d8745f54-fv6nt" event={"ID":"0b146bba-088f-4f7f-b687-20fda1fa2e4f","Type":"ContainerDied","Data":"9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452"} Jul 22 03:47:03.717865 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.717806 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d8745f54-fv6nt" event={"ID":"0b146bba-088f-4f7f-b687-20fda1fa2e4f","Type":"ContainerDied","Data":"dc1b4e97f701ab4fdababdd11a071c55e0f0fbf5e942211a717fa5a88745e44b"} Jul 22 03:47:03.717865 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.717831 2564 scope.go:117] "RemoveContainer" containerID="9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452" Jul 22 03:47:03.726713 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.726697 2564 scope.go:117] "RemoveContainer" containerID="9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452" Jul 22 03:47:03.726973 ip-10-0-139-52 kubenswrapper[2564]: E0722 03:47:03.726956 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452\": container with ID starting with 9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452 not found: ID does not exist" containerID="9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452" Jul 22 03:47:03.727017 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.726981 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452"} err="failed to get container status \"9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452\": rpc error: code = NotFound desc = could not find container \"9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452\": container with ID starting with 9f0a68cc0056ff7b713d6a496ba7f194ebcb04b6d26157022673fe01e811e452 not found: ID does not exist" Jul 22 03:47:03.752410 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.752383 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74d8745f54-fv6nt"] Jul 22 03:47:03.752410 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:03.752412 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74d8745f54-fv6nt"] Jul 22 03:47:04.524538 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.524504 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74d8745f54-ng7b4"] Jul 22 03:47:04.524956 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.524843 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b146bba-088f-4f7f-b687-20fda1fa2e4f" containerName="manager" Jul 22 03:47:04.524956 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.524854 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b146bba-088f-4f7f-b687-20fda1fa2e4f" containerName="manager" Jul 22 03:47:04.524956 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.524927 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b146bba-088f-4f7f-b687-20fda1fa2e4f" containerName="manager" Jul 22 03:47:04.529162 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.529145 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.533833 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.533642 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 03:47:04.533833 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.533686 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 03:47:04.533833 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.533768 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-fkk4q\"" Jul 22 03:47:04.537394 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.537373 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d8745f54-ng7b4"] Jul 22 03:47:04.615687 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.615647 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wlrm\" (UniqueName: \"kubernetes.io/projected/530f4bcb-7050-41f0-ac8d-47f44b676a7b-kube-api-access-2wlrm\") pod \"maas-controller-74d8745f54-ng7b4\" (UID: \"530f4bcb-7050-41f0-ac8d-47f44b676a7b\") " pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.615855 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.615755 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/530f4bcb-7050-41f0-ac8d-47f44b676a7b-webhook-cert\") pod \"maas-controller-74d8745f54-ng7b4\" (UID: \"530f4bcb-7050-41f0-ac8d-47f44b676a7b\") " pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.716610 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.716570 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wlrm\" (UniqueName: \"kubernetes.io/projected/530f4bcb-7050-41f0-ac8d-47f44b676a7b-kube-api-access-2wlrm\") pod \"maas-controller-74d8745f54-ng7b4\" (UID: \"530f4bcb-7050-41f0-ac8d-47f44b676a7b\") " pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.716760 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.716657 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/530f4bcb-7050-41f0-ac8d-47f44b676a7b-webhook-cert\") pod \"maas-controller-74d8745f54-ng7b4\" (UID: \"530f4bcb-7050-41f0-ac8d-47f44b676a7b\") " pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.719414 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.719389 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/530f4bcb-7050-41f0-ac8d-47f44b676a7b-webhook-cert\") pod \"maas-controller-74d8745f54-ng7b4\" (UID: \"530f4bcb-7050-41f0-ac8d-47f44b676a7b\") " pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.730395 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.730364 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wlrm\" (UniqueName: \"kubernetes.io/projected/530f4bcb-7050-41f0-ac8d-47f44b676a7b-kube-api-access-2wlrm\") pod \"maas-controller-74d8745f54-ng7b4\" (UID: \"530f4bcb-7050-41f0-ac8d-47f44b676a7b\") " pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.841112 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.841024 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:04.978905 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:04.978869 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74d8745f54-ng7b4"] Jul 22 03:47:04.979974 ip-10-0-139-52 kubenswrapper[2564]: W0722 03:47:04.979950 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod530f4bcb_7050_41f0_ac8d_47f44b676a7b.slice/crio-3ecf1fc862db33325b0be320b8ded59913eb3aa187fc80b832c5e934e22be5c7 WatchSource:0}: Error finding container 3ecf1fc862db33325b0be320b8ded59913eb3aa187fc80b832c5e934e22be5c7: Status 404 returned error can't find the container with id 3ecf1fc862db33325b0be320b8ded59913eb3aa187fc80b832c5e934e22be5c7 Jul 22 03:47:05.730917 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:05.730834 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d8745f54-ng7b4" event={"ID":"530f4bcb-7050-41f0-ac8d-47f44b676a7b","Type":"ContainerStarted","Data":"27d470bb426dfc2d08e820f96385cad4d94be0e3cc5d2b07aae2156125fb04bb"} Jul 22 03:47:05.730917 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:05.730889 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74d8745f54-ng7b4" event={"ID":"530f4bcb-7050-41f0-ac8d-47f44b676a7b","Type":"ContainerStarted","Data":"3ecf1fc862db33325b0be320b8ded59913eb3aa187fc80b832c5e934e22be5c7"} Jul 22 03:47:05.731295 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:05.730931 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 03:47:05.744248 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:05.744221 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b146bba-088f-4f7f-b687-20fda1fa2e4f" path="/var/lib/kubelet/pods/0b146bba-088f-4f7f-b687-20fda1fa2e4f/volumes" Jul 22 03:47:05.749009 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:05.748962 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74d8745f54-ng7b4" podStartSLOduration=1.353667275 podStartE2EDuration="1.748948327s" totalImagesPullingTime="395.281052ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:47:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:47:05.746684771 +0000 UTC m=+812.540173264" watchObservedRunningTime="2026-07-22 03:47:05.748948327 +0000 UTC m=+812.542436819" Jul 22 03:47:16.738134 ip-10-0-139-52 kubenswrapper[2564]: I0722 03:47:16.738105 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74d8745f54-ng7b4" Jul 22 04:00:00.145765 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:00.145719 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq"] Jul 22 04:00:00.149142 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:00.149125 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq" Jul 22 04:00:00.152565 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:00.152543 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\"" Jul 22 04:00:00.159097 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:00.159078 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq" Jul 22 04:00:00.171298 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:00.171273 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq"] Jul 22 04:00:00.501398 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:00.501373 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq"] Jul 22 04:00:00.503383 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:00:00.503353 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1230403_f01c_4743_96e8_54364c98fd76.slice/crio-91ae68484485b5ed5f7e3b8eebf87c43d95b35b987da30e9fb9b65a7ee93f4d1 WatchSource:0}: Error finding container 91ae68484485b5ed5f7e3b8eebf87c43d95b35b987da30e9fb9b65a7ee93f4d1: Status 404 returned error can't find the container with id 91ae68484485b5ed5f7e3b8eebf87c43d95b35b987da30e9fb9b65a7ee93f4d1 Jul 22 04:00:00.669788 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:00.669756 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq" event={"ID":"a1230403-f01c-4743-96e8-54364c98fd76","Type":"ContainerStarted","Data":"91ae68484485b5ed5f7e3b8eebf87c43d95b35b987da30e9fb9b65a7ee93f4d1"} Jul 22 04:00:01.676657 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:01.676580 2564 generic.go:356] "Generic (PLEG): container finished" podID="a1230403-f01c-4743-96e8-54364c98fd76" containerID="f09f4044a5299cdf472396ca1236fbf0d9d8b9a8feea1eb045f9ac737e2753dd" exitCode=0 Jul 22 04:00:01.676657 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:01.676640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq" event={"ID":"a1230403-f01c-4743-96e8-54364c98fd76","Type":"ContainerDied","Data":"f09f4044a5299cdf472396ca1236fbf0d9d8b9a8feea1eb045f9ac737e2753dd"} Jul 22 04:00:02.813771 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:02.813741 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq" Jul 22 04:00:03.689213 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:03.689175 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq" Jul 22 04:00:03.689425 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:03.689213 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744880-n5bdq" event={"ID":"a1230403-f01c-4743-96e8-54364c98fd76","Type":"ContainerDied","Data":"91ae68484485b5ed5f7e3b8eebf87c43d95b35b987da30e9fb9b65a7ee93f4d1"} Jul 22 04:00:03.689425 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:03.689271 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="91ae68484485b5ed5f7e3b8eebf87c43d95b35b987da30e9fb9b65a7ee93f4d1" Jul 22 04:00:04.753060 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.753027 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk"] Jul 22 04:00:04.753447 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.753390 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1230403-f01c-4743-96e8-54364c98fd76" containerName="cleanup" Jul 22 04:00:04.753447 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.753402 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1230403-f01c-4743-96e8-54364c98fd76" containerName="cleanup" Jul 22 04:00:04.753525 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.753472 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1230403-f01c-4743-96e8-54364c98fd76" containerName="cleanup" Jul 22 04:00:04.756344 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.756315 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.760157 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.760138 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-8ed90c1e-openshift-default-dockercfg-hmqxf\"" Jul 22 04:00:04.771003 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.770981 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk"] Jul 22 04:00:04.882224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882190 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-data\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882231 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-podinfo\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882461 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882362 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df3d3730-cac4-4bfc-a7a0-095e94546271-istiod-ca-cert\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882461 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882400 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-socket\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882461 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882424 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-certs\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882461 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882458 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-envoy\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882588 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882491 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-credential-socket\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882588 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882535 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-token\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.882588 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.882580 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w2zdm\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-kube-api-access-w2zdm\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.983905 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.983870 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w2zdm\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-kube-api-access-w2zdm\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984079 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.983941 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-data\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984079 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.983968 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-podinfo\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984079 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984017 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df3d3730-cac4-4bfc-a7a0-095e94546271-istiod-ca-cert\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984079 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984046 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-socket\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984079 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984073 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-certs\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984361 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984118 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-envoy\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984361 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984154 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-credential-socket\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984361 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984197 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-token\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984530 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984434 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-data\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984585 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984551 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-certs\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984920 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984827 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-credential-socket\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.984920 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984878 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-socket\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.985252 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.984916 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df3d3730-cac4-4bfc-a7a0-095e94546271-istiod-ca-cert\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.986733 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.986711 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-envoy\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.987027 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.987007 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-podinfo\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:04.998977 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:04.998956 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w2zdm\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-kube-api-access-w2zdm\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:05.002862 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.002846 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-token\") pod \"e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:05.068368 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.068301 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:05.239207 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.239181 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk"] Jul 22 04:00:05.239367 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:00:05.239317 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3d3730_cac4_4bfc_a7a0_095e94546271.slice/crio-385f48fb4ad02ab94fecf2f71908004041a6d78cfe41633d54941115e7ed0a9e WatchSource:0}: Error finding container 385f48fb4ad02ab94fecf2f71908004041a6d78cfe41633d54941115e7ed0a9e: Status 404 returned error can't find the container with id 385f48fb4ad02ab94fecf2f71908004041a6d78cfe41633d54941115e7ed0a9e Jul 22 04:00:05.241508 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.241477 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:05.241572 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.241542 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:05.241608 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.241570 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:05.701108 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.701074 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" event={"ID":"df3d3730-cac4-4bfc-a7a0-095e94546271","Type":"ContainerStarted","Data":"c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87"} Jul 22 04:00:05.701284 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.701126 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" event={"ID":"df3d3730-cac4-4bfc-a7a0-095e94546271","Type":"ContainerStarted","Data":"385f48fb4ad02ab94fecf2f71908004041a6d78cfe41633d54941115e7ed0a9e"} Jul 22 04:00:05.727108 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:05.727060 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" podStartSLOduration=1.727040039 podStartE2EDuration="1.727040039s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:00:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:00:05.725152934 +0000 UTC m=+1592.518641426" watchObservedRunningTime="2026-07-22 04:00:05.727040039 +0000 UTC m=+1592.520528533" Jul 22 04:00:06.068575 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:06.068546 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:06.073867 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:06.073846 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:06.707071 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:06.707040 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:06.707993 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:06.707973 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:22.373849 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.373778 2564 patch_prober.go:28] interesting pod/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": read tcp 10.133.0.2:57858->10.133.0.46:15021: read: connection reset by peer" start-of-body= Jul 22 04:00:22.374244 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.373874 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": read tcp 10.133.0.2:57858->10.133.0.46:15021: read: connection reset by peer" Jul 22 04:00:22.795297 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.795260 2564 generic.go:356] "Generic (PLEG): container finished" podID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerID="c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87" exitCode=0 Jul 22 04:00:22.795455 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.795350 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" event={"ID":"df3d3730-cac4-4bfc-a7a0-095e94546271","Type":"ContainerDied","Data":"c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87"} Jul 22 04:00:22.795758 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.795741 2564 scope.go:117] "RemoveContainer" containerID="c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87" Jul 22 04:00:22.796734 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.796704 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:22.796803 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.796785 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:22.796849 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:22.796833 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:23.808109 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:23.808064 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" event={"ID":"df3d3730-cac4-4bfc-a7a0-095e94546271","Type":"ContainerStarted","Data":"28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd"} Jul 22 04:00:24.068454 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:24.068368 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:24.068654 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:24.068626 2564 patch_prober.go:28] interesting pod/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 04:00:24.068734 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:24.068685 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 04:00:25.069380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:25.069321 2564 patch_prober.go:28] interesting pod/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 04:00:25.069786 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:25.069397 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 04:00:26.069308 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:26.069275 2564 patch_prober.go:28] interesting pod/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 04:00:26.069500 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:26.069326 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 04:00:27.068627 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:27.068587 2564 patch_prober.go:28] interesting pod/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Jul 22 04:00:27.068815 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:27.068640 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Jul 22 04:00:28.071797 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:28.071766 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:28.072244 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:28.072230 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:28.072840 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:28.072822 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:32.687549 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.687517 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c"] Jul 22 04:00:32.690807 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.690789 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" Jul 22 04:00:32.693961 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.693941 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\"" Jul 22 04:00:32.694366 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.694247 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 04:00:32.702557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.702523 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c"] Jul 22 04:00:32.834266 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.834234 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a7ac76-9c8d-4cd2-a70e-1239fc3490bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c\" (UID: \"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" Jul 22 04:00:32.935549 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.935514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a7ac76-9c8d-4cd2-a70e-1239fc3490bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c\" (UID: \"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" Jul 22 04:00:32.936195 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:32.936177 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a7ac76-9c8d-4cd2-a70e-1239fc3490bf-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c\" (UID: \"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" Jul 22 04:00:33.002595 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:33.002571 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" Jul 22 04:00:33.348270 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:33.348234 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c"] Jul 22 04:00:33.349297 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:00:33.349268 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96a7ac76_9c8d_4cd2_a70e_1239fc3490bf.slice/crio-9031174d21ad832f5b6175c0e2b9b76cc51ff0b50bf5e15b4ca6ac378af61b7c WatchSource:0}: Error finding container 9031174d21ad832f5b6175c0e2b9b76cc51ff0b50bf5e15b4ca6ac378af61b7c: Status 404 returned error can't find the container with id 9031174d21ad832f5b6175c0e2b9b76cc51ff0b50bf5e15b4ca6ac378af61b7c Jul 22 04:00:33.860961 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:33.860930 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" event={"ID":"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf","Type":"ContainerStarted","Data":"c7686205ff57b2f226632641025e9df77266dca368fc6a808efde5362d246229"} Jul 22 04:00:33.861272 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:33.860974 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" event={"ID":"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf","Type":"ContainerStarted","Data":"9031174d21ad832f5b6175c0e2b9b76cc51ff0b50bf5e15b4ca6ac378af61b7c"} Jul 22 04:00:33.880024 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:33.879969 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" podStartSLOduration=1.879951814 podStartE2EDuration="1.879951814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:00:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:00:33.879159266 +0000 UTC m=+1620.672647758" watchObservedRunningTime="2026-07-22 04:00:33.879951814 +0000 UTC m=+1620.673440307" Jul 22 04:00:34.867523 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:34.867487 2564 generic.go:356] "Generic (PLEG): container finished" podID="96a7ac76-9c8d-4cd2-a70e-1239fc3490bf" containerID="c7686205ff57b2f226632641025e9df77266dca368fc6a808efde5362d246229" exitCode=0 Jul 22 04:00:34.867882 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:34.867571 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" event={"ID":"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf","Type":"ContainerDied","Data":"c7686205ff57b2f226632641025e9df77266dca368fc6a808efde5362d246229"} Jul 22 04:00:36.007162 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:36.007141 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" Jul 22 04:00:36.061522 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:36.061496 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a7ac76-9c8d-4cd2-a70e-1239fc3490bf-maas-api-service-ca\") pod \"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf\" (UID: \"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf\") " Jul 22 04:00:36.061818 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:36.061796 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96a7ac76-9c8d-4cd2-a70e-1239fc3490bf-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96a7ac76-9c8d-4cd2-a70e-1239fc3490bf" (UID: "96a7ac76-9c8d-4cd2-a70e-1239fc3490bf"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:00:36.162825 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:36.162745 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96a7ac76-9c8d-4cd2-a70e-1239fc3490bf-maas-api-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:36.878787 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:36.878762 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" Jul 22 04:00:36.878787 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:36.878774 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c" event={"ID":"96a7ac76-9c8d-4cd2-a70e-1239fc3490bf","Type":"ContainerDied","Data":"9031174d21ad832f5b6175c0e2b9b76cc51ff0b50bf5e15b4ca6ac378af61b7c"} Jul 22 04:00:36.878989 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:36.878818 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9031174d21ad832f5b6175c0e2b9b76cc51ff0b50bf5e15b4ca6ac378af61b7c" Jul 22 04:00:47.770474 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:47.770446 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c"] Jul 22 04:00:47.773617 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:47.773594 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8ed90c1e-8v84c"] Jul 22 04:00:47.910492 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:47.910461 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk"] Jul 22 04:00:47.910721 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:47.910696 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" containerID="cri-o://28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd" gracePeriod=30 Jul 22 04:00:48.989050 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:48.989016 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2"] Jul 22 04:00:48.989609 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:48.989585 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96a7ac76-9c8d-4cd2-a70e-1239fc3490bf" containerName="revoke-keys" Jul 22 04:00:48.989609 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:48.989606 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="96a7ac76-9c8d-4cd2-a70e-1239fc3490bf" containerName="revoke-keys" Jul 22 04:00:48.989846 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:48.989709 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="96a7ac76-9c8d-4cd2-a70e-1239fc3490bf" containerName="revoke-keys" Jul 22 04:00:48.993163 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:48.993143 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:48.996324 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:48.996306 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-19ab37b5-gw-openshift-default-dockercfg-fp94b\"" Jul 22 04:00:49.005188 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.005167 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2"] Jul 22 04:00:49.065518 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065489 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istiod-ca-cert\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065612 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065523 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-certs\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065612 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-envoy\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065612 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065580 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-token\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065773 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065645 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-podinfo\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065773 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065696 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-socket\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065837 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065764 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-credential-socket\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065837 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065830 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd7ck\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-kube-api-access-cd7ck\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.065903 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.065858 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-data\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.166896 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.166860 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-credential-socket\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167053 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.166913 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd7ck\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-kube-api-access-cd7ck\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167053 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.166948 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-data\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167053 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.166994 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istiod-ca-cert\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167053 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167023 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-certs\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167053 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167049 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-envoy\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167325 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167072 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-token\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167325 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167112 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-podinfo\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167481 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167328 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-credential-socket\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167481 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-socket\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167769 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167742 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-socket\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167865 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167779 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-certs\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167865 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167811 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istiod-ca-cert\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.167865 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.167809 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-data\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.169822 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.169795 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-envoy\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.169935 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.169862 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-podinfo\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.179640 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.179614 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd7ck\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-kube-api-access-cd7ck\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.179724 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.179669 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-token\") pod \"e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.305156 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.305132 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:49.449065 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.449040 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2"] Jul 22 04:00:49.453007 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:00:49.452975 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6f04f3d_a3dc_4eb9_b322_1bc527b75f04.slice/crio-3190d245d2a1ecdcff4e9ad15578c7557302c2bf7fa6aa5a4e90854c4c662d2b WatchSource:0}: Error finding container 3190d245d2a1ecdcff4e9ad15578c7557302c2bf7fa6aa5a4e90854c4c662d2b: Status 404 returned error can't find the container with id 3190d245d2a1ecdcff4e9ad15578c7557302c2bf7fa6aa5a4e90854c4c662d2b Jul 22 04:00:49.454515 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.454482 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:49.454605 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.454547 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:49.454605 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.454577 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:00:49.745311 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.745274 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96a7ac76-9c8d-4cd2-a70e-1239fc3490bf" path="/var/lib/kubelet/pods/96a7ac76-9c8d-4cd2-a70e-1239fc3490bf/volumes" Jul 22 04:00:49.950622 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.950591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" event={"ID":"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04","Type":"ContainerStarted","Data":"c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b"} Jul 22 04:00:49.950796 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.950640 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" event={"ID":"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04","Type":"ContainerStarted","Data":"3190d245d2a1ecdcff4e9ad15578c7557302c2bf7fa6aa5a4e90854c4c662d2b"} Jul 22 04:00:49.977105 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:49.977061 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" podStartSLOduration=1.977048753 podStartE2EDuration="1.977048753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:00:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:00:49.975244541 +0000 UTC m=+1636.768733032" watchObservedRunningTime="2026-07-22 04:00:49.977048753 +0000 UTC m=+1636.770537245" Jul 22 04:00:50.305499 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:50.305472 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:50.310361 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:50.310320 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:50.956151 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:50.956121 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:50.957078 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:50.957056 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:00:53.169773 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.169738 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:53.203138 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203080 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w2zdm\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-kube-api-access-w2zdm\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203138 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203116 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-token\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203321 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203175 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-data\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203321 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203201 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-certs\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203321 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203259 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df3d3730-cac4-4bfc-a7a0-095e94546271-istiod-ca-cert\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203321 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203287 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-socket\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203321 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203314 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-podinfo\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203604 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203382 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-credential-socket\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203604 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203411 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-envoy\") pod \"df3d3730-cac4-4bfc-a7a0-095e94546271\" (UID: \"df3d3730-cac4-4bfc-a7a0-095e94546271\") " Jul 22 04:00:53.203604 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203528 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:00:53.203768 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203744 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:00:53.203899 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203874 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:00:53.203963 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.203944 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.204182 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.204152 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df3d3730-cac4-4bfc-a7a0-095e94546271-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:00:53.204304 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.204222 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-data" (OuterVolumeSpecName: "istio-data") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:00:53.205644 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.205619 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-token" (OuterVolumeSpecName: "istio-token") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:00:53.205932 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.205907 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-kube-api-access-w2zdm" (OuterVolumeSpecName: "kube-api-access-w2zdm") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "kube-api-access-w2zdm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:00:53.206106 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.206085 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:00:53.206179 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.206127 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "df3d3730-cac4-4bfc-a7a0-095e94546271" (UID: "df3d3730-cac4-4bfc-a7a0-095e94546271"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:00:53.305078 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305053 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.305078 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305078 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/df3d3730-cac4-4bfc-a7a0-095e94546271-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.305253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305091 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.305253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305102 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.305253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305114 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.305253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305126 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.305253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305139 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w2zdm\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-kube-api-access-w2zdm\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.305253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.305150 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/df3d3730-cac4-4bfc-a7a0-095e94546271-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:00:53.978828 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.978792 2564 generic.go:356] "Generic (PLEG): container finished" podID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerID="28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd" exitCode=0 Jul 22 04:00:53.978978 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.978853 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" Jul 22 04:00:53.978978 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.978877 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" event={"ID":"df3d3730-cac4-4bfc-a7a0-095e94546271","Type":"ContainerDied","Data":"28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd"} Jul 22 04:00:53.978978 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.978961 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk" event={"ID":"df3d3730-cac4-4bfc-a7a0-095e94546271","Type":"ContainerDied","Data":"385f48fb4ad02ab94fecf2f71908004041a6d78cfe41633d54941115e7ed0a9e"} Jul 22 04:00:53.979086 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.978983 2564 scope.go:117] "RemoveContainer" containerID="28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd" Jul 22 04:00:53.988071 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.988049 2564 scope.go:117] "RemoveContainer" containerID="c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87" Jul 22 04:00:53.996174 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.996157 2564 scope.go:117] "RemoveContainer" containerID="28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd" Jul 22 04:00:53.996460 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:00:53.996441 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd\": container with ID starting with 28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd not found: ID does not exist" containerID="28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd" Jul 22 04:00:53.996540 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.996472 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd"} err="failed to get container status \"28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd\": rpc error: code = NotFound desc = could not find container \"28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd\": container with ID starting with 28af1534d0a7b5486708a72907e8f6c300b909915d1b49a2d4a74dab8080addd not found: ID does not exist" Jul 22 04:00:53.996540 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.996495 2564 scope.go:117] "RemoveContainer" containerID="c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87" Jul 22 04:00:53.996785 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:00:53.996768 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87\": container with ID starting with c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87 not found: ID does not exist" containerID="c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87" Jul 22 04:00:53.996829 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:53.996792 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87"} err="failed to get container status \"c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87\": rpc error: code = NotFound desc = could not find container \"c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87\": container with ID starting with c79005f1dbc916974b4f37122fddc8b6124f6a0c9872c7979802865869009c87 not found: ID does not exist" Jul 22 04:00:54.002465 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:54.002414 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk"] Jul 22 04:00:54.006694 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:54.006670 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-8ed90c1e-openshift-default-95dd8c577-mdkqk"] Jul 22 04:00:55.745461 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:00:55.745430 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" path="/var/lib/kubelet/pods/df3d3730-cac4-4bfc-a7a0-095e94546271/volumes" Jul 22 04:01:05.958378 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:05.958256 2564 patch_prober.go:28] interesting pod/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 04:01:05.958378 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:05.958322 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 04:01:06.046166 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:06.046131 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerID="c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b" exitCode=0 Jul 22 04:01:06.046366 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:06.046204 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" event={"ID":"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04","Type":"ContainerDied","Data":"c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b"} Jul 22 04:01:06.046613 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:06.046599 2564 scope.go:117] "RemoveContainer" containerID="c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b" Jul 22 04:01:06.047611 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:06.047580 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:06.047709 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:06.047660 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:06.047709 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:06.047693 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:07.053590 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:07.053552 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" event={"ID":"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04","Type":"ContainerStarted","Data":"d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067"} Jul 22 04:01:07.306067 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:07.305966 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:01:07.306225 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:07.306208 2564 patch_prober.go:28] interesting pod/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 04:01:07.306276 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:07.306254 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 04:01:08.306541 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:08.306505 2564 patch_prober.go:28] interesting pod/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 04:01:08.306969 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:08.306576 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 04:01:09.306153 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:09.306116 2564 patch_prober.go:28] interesting pod/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 04:01:09.306328 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:09.306168 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 04:01:10.306128 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:10.306091 2564 patch_prober.go:28] interesting pod/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Jul 22 04:01:10.306546 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:10.306145 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Jul 22 04:01:11.309529 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:11.309502 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:01:11.309992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:11.309769 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:01:11.310442 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:11.310426 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:01:16.489786 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.489754 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9"] Jul 22 04:01:16.490166 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.490124 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" Jul 22 04:01:16.490166 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.490136 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" Jul 22 04:01:16.490239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.490204 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" Jul 22 04:01:16.490239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.490212 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" Jul 22 04:01:16.492529 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.492512 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" Jul 22 04:01:16.501846 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.501825 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\"" Jul 22 04:01:16.501987 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.501847 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 04:01:16.504322 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.504295 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9"] Jul 22 04:01:16.607151 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.607117 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d71e742-3292-4163-8e6b-c7e8513aef45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9\" (UID: \"2d71e742-3292-4163-8e6b-c7e8513aef45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" Jul 22 04:01:16.708103 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.708067 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d71e742-3292-4163-8e6b-c7e8513aef45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9\" (UID: \"2d71e742-3292-4163-8e6b-c7e8513aef45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" Jul 22 04:01:16.708719 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.708700 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d71e742-3292-4163-8e6b-c7e8513aef45-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9\" (UID: \"2d71e742-3292-4163-8e6b-c7e8513aef45\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" Jul 22 04:01:16.803798 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.803763 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" Jul 22 04:01:16.935743 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:16.935713 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9"] Jul 22 04:01:16.936842 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:01:16.936815 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d71e742_3292_4163_8e6b_c7e8513aef45.slice/crio-4cad0aa33009838c8c1ed1262836353d9a66fbf4a829724d84d0669fe13c475f WatchSource:0}: Error finding container 4cad0aa33009838c8c1ed1262836353d9a66fbf4a829724d84d0669fe13c475f: Status 404 returned error can't find the container with id 4cad0aa33009838c8c1ed1262836353d9a66fbf4a829724d84d0669fe13c475f Jul 22 04:01:17.107720 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:17.107639 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" event={"ID":"2d71e742-3292-4163-8e6b-c7e8513aef45","Type":"ContainerStarted","Data":"4cad0aa33009838c8c1ed1262836353d9a66fbf4a829724d84d0669fe13c475f"} Jul 22 04:01:18.114868 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:18.114787 2564 generic.go:356] "Generic (PLEG): container finished" podID="2d71e742-3292-4163-8e6b-c7e8513aef45" containerID="3f48af870155160701dc50ae2e451a296c473512ab468a0164c69d85b86faee3" exitCode=0 Jul 22 04:01:18.114868 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:18.114845 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" event={"ID":"2d71e742-3292-4163-8e6b-c7e8513aef45","Type":"ContainerDied","Data":"3f48af870155160701dc50ae2e451a296c473512ab468a0164c69d85b86faee3"} Jul 22 04:01:19.254634 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:19.254608 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" Jul 22 04:01:19.329932 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:19.329896 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d71e742-3292-4163-8e6b-c7e8513aef45-maas-api-service-ca\") pod \"2d71e742-3292-4163-8e6b-c7e8513aef45\" (UID: \"2d71e742-3292-4163-8e6b-c7e8513aef45\") " Jul 22 04:01:19.330261 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:19.330234 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d71e742-3292-4163-8e6b-c7e8513aef45-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2d71e742-3292-4163-8e6b-c7e8513aef45" (UID: "2d71e742-3292-4163-8e6b-c7e8513aef45"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:01:19.431234 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:19.431148 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d71e742-3292-4163-8e6b-c7e8513aef45-maas-api-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:20.127092 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:20.127003 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" Jul 22 04:01:20.127092 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:20.127045 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9" event={"ID":"2d71e742-3292-4163-8e6b-c7e8513aef45","Type":"ContainerDied","Data":"4cad0aa33009838c8c1ed1262836353d9a66fbf4a829724d84d0669fe13c475f"} Jul 22 04:01:20.127092 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:20.127092 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4cad0aa33009838c8c1ed1262836353d9a66fbf4a829724d84d0669fe13c475f" Jul 22 04:01:31.610939 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:31.610900 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9"] Jul 22 04:01:31.614050 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:31.614025 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-19ab37b5-lblm9"] Jul 22 04:01:31.747238 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:31.747205 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d71e742-3292-4163-8e6b-c7e8513aef45" path="/var/lib/kubelet/pods/2d71e742-3292-4163-8e6b-c7e8513aef45/volumes" Jul 22 04:01:31.771123 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:31.771093 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2"] Jul 22 04:01:31.771380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:31.771357 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" containerID="cri-o://d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067" gracePeriod=30 Jul 22 04:01:32.573081 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.573051 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b"] Jul 22 04:01:32.573468 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.573454 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d71e742-3292-4163-8e6b-c7e8513aef45" containerName="revoke-keys" Jul 22 04:01:32.573521 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.573470 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d71e742-3292-4163-8e6b-c7e8513aef45" containerName="revoke-keys" Jul 22 04:01:32.573521 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.573480 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" Jul 22 04:01:32.573521 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.573485 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="df3d3730-cac4-4bfc-a7a0-095e94546271" containerName="istio-proxy" Jul 22 04:01:32.573613 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.573561 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d71e742-3292-4163-8e6b-c7e8513aef45" containerName="revoke-keys" Jul 22 04:01:32.576485 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.576470 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.579481 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.579458 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a1bed135-openshift-default-dockercfg-tt9t8\"" Jul 22 04:01:32.590875 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.590853 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b"] Jul 22 04:01:32.755918 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.755880 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-certs\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.755918 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.755917 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-socket\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.756375 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.755943 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdw5x\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-kube-api-access-hdw5x\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.756375 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.755994 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-podinfo\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.756375 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.756036 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-data\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.756375 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.756081 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d810d555-cbe8-410b-9e95-75901aba0f0d-istiod-ca-cert\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.756375 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.756123 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-credential-socket\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.756375 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.756143 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-envoy\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.756375 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.756160 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-token\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.857873 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.857755 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-data\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.858102 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858082 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d810d555-cbe8-410b-9e95-75901aba0f0d-istiod-ca-cert\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.858309 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858132 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-data\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.858460 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858287 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-credential-socket\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.858460 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858385 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-envoy\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.858460 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858414 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-token\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.858728 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858699 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-certs\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.858918 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858885 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-socket\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.859027 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858961 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdw5x\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-kube-api-access-hdw5x\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.859027 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.858997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-podinfo\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.859313 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.859292 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-credential-socket\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.859528 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.859507 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-certs\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.859613 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.859558 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d810d555-cbe8-410b-9e95-75901aba0f0d-istiod-ca-cert\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.859744 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.859714 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-socket\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.862030 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.862003 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-envoy\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.862160 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.862013 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-podinfo\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.875626 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.875601 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdw5x\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-kube-api-access-hdw5x\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.876822 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.876798 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-token\") pod \"e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:32.887228 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:32.887205 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:33.033271 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:01:33.033242 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd810d555_cbe8_410b_9e95_75901aba0f0d.slice/crio-b573b7a37392b913a2753d899fc065223b84e4dab26d39e29af8869305ffce50 WatchSource:0}: Error finding container b573b7a37392b913a2753d899fc065223b84e4dab26d39e29af8869305ffce50: Status 404 returned error can't find the container with id b573b7a37392b913a2753d899fc065223b84e4dab26d39e29af8869305ffce50 Jul 22 04:01:33.033854 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.033785 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b"] Jul 22 04:01:33.035300 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.035271 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:33.035386 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.035353 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:33.035424 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.035390 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:33.200101 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.200010 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" event={"ID":"d810d555-cbe8-410b-9e95-75901aba0f0d","Type":"ContainerStarted","Data":"81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73"} Jul 22 04:01:33.200101 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.200070 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" event={"ID":"d810d555-cbe8-410b-9e95-75901aba0f0d","Type":"ContainerStarted","Data":"b573b7a37392b913a2753d899fc065223b84e4dab26d39e29af8869305ffce50"} Jul 22 04:01:33.242559 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.242498 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" podStartSLOduration=1.242477729 podStartE2EDuration="1.242477729s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:01:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:01:33.238728098 +0000 UTC m=+1680.032216590" watchObservedRunningTime="2026-07-22 04:01:33.242477729 +0000 UTC m=+1680.035966221" Jul 22 04:01:33.887505 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:33.887465 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:34.893310 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:34.893278 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:35.211782 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:35.211697 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:35.212746 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:35.212722 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:37.019732 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.019702 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:01:37.200285 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200200 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-socket\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.200285 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200257 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-token\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.200285 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200281 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-podinfo\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.200573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200326 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-certs\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.200573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200364 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd7ck\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-kube-api-access-cd7ck\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.200573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200405 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-data\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.200573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200437 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istiod-ca-cert\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.200573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200516 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:01:37.200837 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200633 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:01:37.200892 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.200854 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:01:37.201167 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201105 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-credential-socket\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.201167 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201148 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-envoy\") pod \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\" (UID: \"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04\") " Jul 22 04:01:37.201385 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201254 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-data" (OuterVolumeSpecName: "istio-data") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:01:37.201460 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201386 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:01:37.201668 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201648 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.201738 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201676 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.201738 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201692 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.201738 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201708 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.201738 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.201721 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.202761 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.202731 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-kube-api-access-cd7ck" (OuterVolumeSpecName: "kube-api-access-cd7ck") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "kube-api-access-cd7ck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:01:37.202761 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.202747 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-token" (OuterVolumeSpecName: "istio-token") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:01:37.202873 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.202854 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:01:37.203285 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.203267 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" (UID: "e6f04f3d-a3dc-4eb9-b322-1bc527b75f04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:01:37.222785 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.222761 2564 generic.go:356] "Generic (PLEG): container finished" podID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerID="d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067" exitCode=0 Jul 22 04:01:37.222885 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.222819 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" Jul 22 04:01:37.222885 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.222858 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" event={"ID":"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04","Type":"ContainerDied","Data":"d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067"} Jul 22 04:01:37.223000 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.222955 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2" event={"ID":"e6f04f3d-a3dc-4eb9-b322-1bc527b75f04","Type":"ContainerDied","Data":"3190d245d2a1ecdcff4e9ad15578c7557302c2bf7fa6aa5a4e90854c4c662d2b"} Jul 22 04:01:37.223000 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.222986 2564 scope.go:117] "RemoveContainer" containerID="d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067" Jul 22 04:01:37.232390 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.232374 2564 scope.go:117] "RemoveContainer" containerID="c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b" Jul 22 04:01:37.240191 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.240176 2564 scope.go:117] "RemoveContainer" containerID="d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067" Jul 22 04:01:37.240430 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:01:37.240414 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067\": container with ID starting with d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067 not found: ID does not exist" containerID="d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067" Jul 22 04:01:37.240496 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.240437 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067"} err="failed to get container status \"d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067\": rpc error: code = NotFound desc = could not find container \"d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067\": container with ID starting with d995d6ca0ce3b8167822eed57feb7cbdbf36fff03e089c20d1583d103e3f6067 not found: ID does not exist" Jul 22 04:01:37.240496 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.240453 2564 scope.go:117] "RemoveContainer" containerID="c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b" Jul 22 04:01:37.240661 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:01:37.240647 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b\": container with ID starting with c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b not found: ID does not exist" containerID="c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b" Jul 22 04:01:37.240701 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.240664 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b"} err="failed to get container status \"c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b\": rpc error: code = NotFound desc = could not find container \"c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b\": container with ID starting with c8c2d42c74ccbab6da76a1a94bf2fca2941780fa1d82430810316d8cbc4a661b not found: ID does not exist" Jul 22 04:01:37.248062 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.248038 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2"] Jul 22 04:01:37.251980 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.251963 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-19ab37b5-gw-openshift-default-656fc769f-gphr2"] Jul 22 04:01:37.302224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.302204 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd7ck\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-kube-api-access-cd7ck\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.302324 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.302229 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.302324 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.302243 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.302324 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.302255 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:01:37.745913 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:37.745882 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" path="/var/lib/kubelet/pods/e6f04f3d-a3dc-4eb9-b322-1bc527b75f04/volumes" Jul 22 04:01:50.213564 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.213527 2564 patch_prober.go:28] interesting pod/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 04:01:50.214006 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.213600 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 04:01:50.302231 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.300965 2564 generic.go:356] "Generic (PLEG): container finished" podID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerID="81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73" exitCode=0 Jul 22 04:01:50.302231 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.301045 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" event={"ID":"d810d555-cbe8-410b-9e95-75901aba0f0d","Type":"ContainerDied","Data":"81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73"} Jul 22 04:01:50.302231 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.301525 2564 scope.go:117] "RemoveContainer" containerID="81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73" Jul 22 04:01:50.303573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.303529 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:50.303709 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.303643 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:50.303781 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:50.303707 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:01:51.308369 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:51.308316 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" event={"ID":"d810d555-cbe8-410b-9e95-75901aba0f0d","Type":"ContainerStarted","Data":"422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83"} Jul 22 04:01:51.890363 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:51.889484 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:51.890363 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:51.889883 2564 patch_prober.go:28] interesting pod/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 04:01:51.890363 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:51.889930 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 04:01:52.890226 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:52.888713 2564 patch_prober.go:28] interesting pod/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 04:01:52.890226 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:52.888769 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 04:01:53.888227 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:53.888189 2564 patch_prober.go:28] interesting pod/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 04:01:53.888542 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:53.888508 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 04:01:54.888099 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:54.888055 2564 patch_prober.go:28] interesting pod/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body= Jul 22 04:01:54.888610 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:54.888121 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" Jul 22 04:01:55.892024 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:55.891978 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:55.892536 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:55.892279 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:01:55.893035 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:01:55.893011 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:02:16.773499 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:16.773463 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b"] Jul 22 04:02:16.774118 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:16.773717 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" containerID="cri-o://422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83" gracePeriod=30 Jul 22 04:02:17.551266 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.551226 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc"] Jul 22 04:02:17.551679 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.551660 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" Jul 22 04:02:17.551774 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.551682 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" Jul 22 04:02:17.551774 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.551708 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" Jul 22 04:02:17.551774 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.551718 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" Jul 22 04:02:17.551927 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.551830 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" Jul 22 04:02:17.551927 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.551846 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="e6f04f3d-a3dc-4eb9-b322-1bc527b75f04" containerName="istio-proxy" Jul 22 04:02:17.554412 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.554392 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.557415 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.557394 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-9593c364-openshift-default-dockercfg-gt6v4\"" Jul 22 04:02:17.569681 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.569660 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc"] Jul 22 04:02:17.635644 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.635617 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-podinfo\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.635941 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.635898 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-credential-socket\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.636122 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.636106 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjrlb\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-kube-api-access-kjrlb\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.636302 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.636277 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-socket\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.636453 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.636312 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-envoy\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.636453 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.636367 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-data\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.636577 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.636461 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-token\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.636640 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.636578 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-certs\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.636727 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.636711 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/400dbe82-5b7d-4c15-bb3d-458caeee8513-istiod-ca-cert\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738004 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.737972 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-podinfo\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738027 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-credential-socket\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738123 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjrlb\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-kube-api-access-kjrlb\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738164 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-socket\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738187 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-envoy\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738473 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738208 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-data\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738473 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738225 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-token\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738473 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738275 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-certs\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738473 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738386 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/400dbe82-5b7d-4c15-bb3d-458caeee8513-istiod-ca-cert\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738683 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-credential-socket\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738683 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738581 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-socket\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738683 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738645 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-data\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.738894 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.738796 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-certs\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.739053 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.739031 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/400dbe82-5b7d-4c15-bb3d-458caeee8513-istiod-ca-cert\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.740760 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.740732 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-podinfo\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.740884 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.740868 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-envoy\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.749061 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.749035 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjrlb\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-kube-api-access-kjrlb\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.749827 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.749807 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-token\") pod \"e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:17.864735 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:17.864677 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:18.007840 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.007813 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc"] Jul 22 04:02:18.008932 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:02:18.008908 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400dbe82_5b7d_4c15_bb3d_458caeee8513.slice/crio-cbc65838e92c219bc3f7da66ada669a8815f11618714bcbf237de5c33477d292 WatchSource:0}: Error finding container cbc65838e92c219bc3f7da66ada669a8815f11618714bcbf237de5c33477d292: Status 404 returned error can't find the container with id cbc65838e92c219bc3f7da66ada669a8815f11618714bcbf237de5c33477d292 Jul 22 04:02:18.011231 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.011202 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:02:18.011297 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.011265 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:02:18.011297 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.011291 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:02:18.464073 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.464039 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" event={"ID":"400dbe82-5b7d-4c15-bb3d-458caeee8513","Type":"ContainerStarted","Data":"993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a"} Jul 22 04:02:18.464260 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.464103 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" event={"ID":"400dbe82-5b7d-4c15-bb3d-458caeee8513","Type":"ContainerStarted","Data":"cbc65838e92c219bc3f7da66ada669a8815f11618714bcbf237de5c33477d292"} Jul 22 04:02:18.485890 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.485823 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" podStartSLOduration=1.485805318 podStartE2EDuration="1.485805318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:02:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:02:18.485168623 +0000 UTC m=+1725.278657117" watchObservedRunningTime="2026-07-22 04:02:18.485805318 +0000 UTC m=+1725.279293810" Jul 22 04:02:18.865612 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:18.865584 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:19.870903 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:19.870874 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:20.474611 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:20.474586 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:20.475505 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:20.475485 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:22.028193 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.028171 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:02:22.174459 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174388 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-certs\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174459 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174429 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-podinfo\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174459 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174453 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-data\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174706 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174590 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-token\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174706 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174658 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdw5x\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-kube-api-access-hdw5x\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174706 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174692 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:02:22.174706 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174705 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-credential-socket\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174908 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174752 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-envoy\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174908 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174791 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-socket\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.174908 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174829 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d810d555-cbe8-410b-9e95-75901aba0f0d-istiod-ca-cert\") pod \"d810d555-cbe8-410b-9e95-75901aba0f0d\" (UID: \"d810d555-cbe8-410b-9e95-75901aba0f0d\") " Jul 22 04:02:22.175072 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.174966 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:02:22.175148 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.175123 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:02:22.175210 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.175170 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.175210 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.175190 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.175493 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.175468 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-data" (OuterVolumeSpecName: "istio-data") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:02:22.175756 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.175542 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d810d555-cbe8-410b-9e95-75901aba0f0d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:02:22.176870 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.176849 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:02:22.177098 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.177075 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:02:22.177273 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.177246 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-kube-api-access-hdw5x" (OuterVolumeSpecName: "kube-api-access-hdw5x") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "kube-api-access-hdw5x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:02:22.177273 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.177254 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-token" (OuterVolumeSpecName: "istio-token") pod "d810d555-cbe8-410b-9e95-75901aba0f0d" (UID: "d810d555-cbe8-410b-9e95-75901aba0f0d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:02:22.275843 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.275820 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.275843 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.275841 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdw5x\" (UniqueName: \"kubernetes.io/projected/d810d555-cbe8-410b-9e95-75901aba0f0d-kube-api-access-hdw5x\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.275969 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.275850 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.275969 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.275858 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.275969 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.275866 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d810d555-cbe8-410b-9e95-75901aba0f0d-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.275969 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.275874 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.275969 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.275882 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d810d555-cbe8-410b-9e95-75901aba0f0d-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:02:22.488215 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.488148 2564 generic.go:356] "Generic (PLEG): container finished" podID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerID="422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83" exitCode=0 Jul 22 04:02:22.488215 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.488201 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" Jul 22 04:02:22.488398 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.488232 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" event={"ID":"d810d555-cbe8-410b-9e95-75901aba0f0d","Type":"ContainerDied","Data":"422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83"} Jul 22 04:02:22.488398 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.488282 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b" event={"ID":"d810d555-cbe8-410b-9e95-75901aba0f0d","Type":"ContainerDied","Data":"b573b7a37392b913a2753d899fc065223b84e4dab26d39e29af8869305ffce50"} Jul 22 04:02:22.488398 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.488306 2564 scope.go:117] "RemoveContainer" containerID="422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83" Jul 22 04:02:22.498431 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.498413 2564 scope.go:117] "RemoveContainer" containerID="81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73" Jul 22 04:02:22.506714 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.506699 2564 scope.go:117] "RemoveContainer" containerID="422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83" Jul 22 04:02:22.506945 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:02:22.506926 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83\": container with ID starting with 422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83 not found: ID does not exist" containerID="422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83" Jul 22 04:02:22.506998 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.506951 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83"} err="failed to get container status \"422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83\": rpc error: code = NotFound desc = could not find container \"422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83\": container with ID starting with 422e561537ddb8d478f5ee61f92966ecd00cbec41b5404f004b703b87a855d83 not found: ID does not exist" Jul 22 04:02:22.506998 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.506966 2564 scope.go:117] "RemoveContainer" containerID="81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73" Jul 22 04:02:22.507197 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:02:22.507183 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73\": container with ID starting with 81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73 not found: ID does not exist" containerID="81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73" Jul 22 04:02:22.507241 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.507201 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73"} err="failed to get container status \"81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73\": rpc error: code = NotFound desc = could not find container \"81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73\": container with ID starting with 81b47441e808ead8c7a00bed713219de2c62a2145c8d60d33fd6050d536abc73 not found: ID does not exist" Jul 22 04:02:22.512817 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.512793 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b"] Jul 22 04:02:22.516565 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:22.516543 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a1bed135-openshift-default-5d7588d695-78j8b"] Jul 22 04:02:23.745487 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:23.745444 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" path="/var/lib/kubelet/pods/d810d555-cbe8-410b-9e95-75901aba0f0d/volumes" Jul 22 04:02:35.747912 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:35.747307 2564 patch_prober.go:28] interesting pod/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:55524->10.133.0.51:15021: read: connection reset by peer" start-of-body= Jul 22 04:02:35.747912 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:35.747388 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": read tcp 10.133.0.2:55524->10.133.0.51:15021: read: connection reset by peer" Jul 22 04:02:36.570005 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:36.569966 2564 generic.go:356] "Generic (PLEG): container finished" podID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerID="993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a" exitCode=0 Jul 22 04:02:36.570177 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:36.570044 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" event={"ID":"400dbe82-5b7d-4c15-bb3d-458caeee8513","Type":"ContainerDied","Data":"993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a"} Jul 22 04:02:36.570459 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:36.570442 2564 scope.go:117] "RemoveContainer" containerID="993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a" Jul 22 04:02:36.571631 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:36.571597 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:02:36.571720 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:36.571678 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:02:36.571720 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:36.571714 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:02:37.577982 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:37.577932 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" event={"ID":"400dbe82-5b7d-4c15-bb3d-458caeee8513","Type":"ContainerStarted","Data":"06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404"} Jul 22 04:02:37.865299 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:37.865207 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:37.865549 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:37.865524 2564 patch_prober.go:28] interesting pod/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 04:02:37.865609 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:37.865590 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 04:02:38.866013 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:38.865979 2564 patch_prober.go:28] interesting pod/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 04:02:38.866422 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:38.866042 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 04:02:39.865252 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:39.865220 2564 patch_prober.go:28] interesting pod/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 04:02:39.865451 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:39.865272 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 04:02:40.865269 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:40.865232 2564 patch_prober.go:28] interesting pod/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Jul 22 04:02:40.865708 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:40.865284 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Jul 22 04:02:41.868583 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:41.868551 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:41.868996 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:41.868814 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:41.869488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:41.869466 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:02:59.988780 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:59.988745 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc"] Jul 22 04:02:59.989314 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:02:59.988996 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" containerID="cri-o://06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404" gracePeriod=30 Jul 22 04:03:05.241098 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.241076 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:03:05.323090 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323034 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjrlb\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-kube-api-access-kjrlb\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323090 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323065 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/400dbe82-5b7d-4c15-bb3d-458caeee8513-istiod-ca-cert\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323263 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323102 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-envoy\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323263 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323119 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-credential-socket\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323263 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323155 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-token\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323263 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323178 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-podinfo\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323263 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323205 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-socket\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323263 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323228 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-data\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323590 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323291 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-certs\") pod \"400dbe82-5b7d-4c15-bb3d-458caeee8513\" (UID: \"400dbe82-5b7d-4c15-bb3d-458caeee8513\") " Jul 22 04:03:05.323590 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323519 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:03:05.323590 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323555 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/400dbe82-5b7d-4c15-bb3d-458caeee8513-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:03:05.323727 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323682 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:03:05.324019 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.323764 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:03:05.324666 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.324638 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-data" (OuterVolumeSpecName: "istio-data") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:03:05.325362 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.325309 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:03:05.325487 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.325467 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-kube-api-access-kjrlb" (OuterVolumeSpecName: "kube-api-access-kjrlb") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "kube-api-access-kjrlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:03:05.325539 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.325499 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-token" (OuterVolumeSpecName: "istio-token") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:03:05.325723 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.325707 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "400dbe82-5b7d-4c15-bb3d-458caeee8513" (UID: "400dbe82-5b7d-4c15-bb3d-458caeee8513"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:03:05.424369 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424322 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424369 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424367 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjrlb\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-kube-api-access-kjrlb\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424378 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/400dbe82-5b7d-4c15-bb3d-458caeee8513-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424386 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424394 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424401 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424409 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424416 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.424488 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.424423 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/400dbe82-5b7d-4c15-bb3d-458caeee8513-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:03:05.735205 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.735147 2564 generic.go:356] "Generic (PLEG): container finished" podID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerID="06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404" exitCode=0 Jul 22 04:03:05.735205 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.735194 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" Jul 22 04:03:05.735325 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.735211 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" event={"ID":"400dbe82-5b7d-4c15-bb3d-458caeee8513","Type":"ContainerDied","Data":"06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404"} Jul 22 04:03:05.735325 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.735251 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc" event={"ID":"400dbe82-5b7d-4c15-bb3d-458caeee8513","Type":"ContainerDied","Data":"cbc65838e92c219bc3f7da66ada669a8815f11618714bcbf237de5c33477d292"} Jul 22 04:03:05.735325 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.735271 2564 scope.go:117] "RemoveContainer" containerID="06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404" Jul 22 04:03:05.745325 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.745306 2564 scope.go:117] "RemoveContainer" containerID="993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a" Jul 22 04:03:05.754998 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.754238 2564 scope.go:117] "RemoveContainer" containerID="06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404" Jul 22 04:03:05.755308 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:03:05.755283 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404\": container with ID starting with 06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404 not found: ID does not exist" containerID="06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404" Jul 22 04:03:05.755406 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.755313 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404"} err="failed to get container status \"06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404\": rpc error: code = NotFound desc = could not find container \"06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404\": container with ID starting with 06e2b2e296ced12a879b683247bd1a1ac85944f90af61c7fc394b8b983868404 not found: ID does not exist" Jul 22 04:03:05.755406 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.755383 2564 scope.go:117] "RemoveContainer" containerID="993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a" Jul 22 04:03:05.755783 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:03:05.755758 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a\": container with ID starting with 993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a not found: ID does not exist" containerID="993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a" Jul 22 04:03:05.755928 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.755904 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a"} err="failed to get container status \"993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a\": rpc error: code = NotFound desc = could not find container \"993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a\": container with ID starting with 993fd74818e078da66ad2ad8880364387d9c0b5529cc49b7c74a2dd5ef39a82a not found: ID does not exist" Jul 22 04:03:05.758799 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.758782 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc"] Jul 22 04:03:05.764768 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:05.764750 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-9593c364-openshift-default-85c74f7b67-h9xqc"] Jul 22 04:03:07.745745 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:03:07.745713 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" path="/var/lib/kubelet/pods/400dbe82-5b7d-4c15-bb3d-458caeee8513/volumes" Jul 22 04:04:43.840209 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840176 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46"] Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840547 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840560 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840574 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840579 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840587 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840593 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840603 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840608 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840675 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840683 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" Jul 22 04:04:43.840702 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.840691 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="d810d555-cbe8-410b-9e95-75901aba0f0d" containerName="istio-proxy" Jul 22 04:04:43.842835 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.842820 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.845872 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.845853 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-9cb7aec7-openshift-default-dockercfg-z6sr2\"" Jul 22 04:04:43.870614 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.870581 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46"] Jul 22 04:04:43.934593 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934560 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-socket\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.934793 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934628 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38346328-e4d0-4419-9e72-2fc3ba02d461-istiod-ca-cert\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.934793 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934661 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-podinfo\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.934793 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934681 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-data\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.934793 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934708 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-token\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.934793 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934763 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-credential-socket\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.935066 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934877 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-certs\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.935066 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.934960 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-envoy\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:43.935066 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:43.935030 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-24l8k\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-kube-api-access-24l8k\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.035979 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.035939 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-socket\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.035979 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.035985 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38346328-e4d0-4419-9e72-2fc3ba02d461-istiod-ca-cert\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036242 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036006 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-podinfo\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036242 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036139 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-data\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036242 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036188 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-token\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036430 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036251 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-credential-socket\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036430 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036294 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-certs\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036430 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036349 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-envoy\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036430 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036403 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-24l8k\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-kube-api-access-24l8k\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036642 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036504 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-socket\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036753 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036720 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-data\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036883 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036801 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-credential-socket\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.036883 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.036821 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38346328-e4d0-4419-9e72-2fc3ba02d461-istiod-ca-cert\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.037061 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.037020 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-certs\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.038814 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.038789 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-podinfo\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.038900 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.038863 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-envoy\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.045653 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.045630 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-token\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.045789 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.045753 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-24l8k\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-kube-api-access-24l8k\") pod \"e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.153410 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.153316 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:44.302952 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.302929 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46"] Jul 22 04:04:44.304465 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:04:44.304440 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38346328_e4d0_4419_9e72_2fc3ba02d461.slice/crio-18d2ebff559d7fd4b8fbb0888cdffae3b58e719380ea77494b24fc018cc9d7ab WatchSource:0}: Error finding container 18d2ebff559d7fd4b8fbb0888cdffae3b58e719380ea77494b24fc018cc9d7ab: Status 404 returned error can't find the container with id 18d2ebff559d7fd4b8fbb0888cdffae3b58e719380ea77494b24fc018cc9d7ab Jul 22 04:04:44.306773 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.306736 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:04:44.306865 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.306816 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:04:44.306933 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:44.306863 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:04:45.242034 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:45.241999 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" event={"ID":"38346328-e4d0-4419-9e72-2fc3ba02d461","Type":"ContainerStarted","Data":"d26807065935cf93352801c8f8da7612ef90a49e53b90caba37431c6f5ba7921"} Jul 22 04:04:45.242410 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:45.242061 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" event={"ID":"38346328-e4d0-4419-9e72-2fc3ba02d461","Type":"ContainerStarted","Data":"18d2ebff559d7fd4b8fbb0888cdffae3b58e719380ea77494b24fc018cc9d7ab"} Jul 22 04:04:45.267491 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:45.267453 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podStartSLOduration=2.267437408 podStartE2EDuration="2.267437408s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:04:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:04:45.264663102 +0000 UTC m=+1872.058151594" watchObservedRunningTime="2026-07-22 04:04:45.267437408 +0000 UTC m=+1872.060925974" Jul 22 04:04:46.154507 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:46.154467 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:46.158998 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:46.158969 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:46.246867 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:46.246831 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:04:46.247670 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:04:46.247651 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:05:02.288536 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:02.288493 2564 patch_prober.go:28] interesting pod/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 04:05:02.288964 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:02.288577 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 04:05:03.345150 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:03.345113 2564 generic.go:356] "Generic (PLEG): container finished" podID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerID="d26807065935cf93352801c8f8da7612ef90a49e53b90caba37431c6f5ba7921" exitCode=0 Jul 22 04:05:03.345600 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:03.345167 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" event={"ID":"38346328-e4d0-4419-9e72-2fc3ba02d461","Type":"ContainerDied","Data":"d26807065935cf93352801c8f8da7612ef90a49e53b90caba37431c6f5ba7921"} Jul 22 04:05:03.345723 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:03.345704 2564 scope.go:117] "RemoveContainer" containerID="d26807065935cf93352801c8f8da7612ef90a49e53b90caba37431c6f5ba7921" Jul 22 04:05:03.346802 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:03.346773 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:03.346868 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:03.346838 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:03.346910 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:03.346874 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:04.352129 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:04.352089 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" event={"ID":"38346328-e4d0-4419-9e72-2fc3ba02d461","Type":"ContainerStarted","Data":"e6e60b02597a3948a18f3c7aa30475882a775cedbfef82b77a2d28cdc2163f1a"} Jul 22 04:05:05.153821 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:05.153781 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:05:05.154058 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:05.154036 2564 patch_prober.go:28] interesting pod/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 04:05:05.154112 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:05.154089 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 04:05:06.154495 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:06.154461 2564 patch_prober.go:28] interesting pod/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 04:05:06.154882 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:06.154516 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 04:05:07.154706 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:07.154671 2564 patch_prober.go:28] interesting pod/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 04:05:07.155093 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:07.154729 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 04:05:08.154573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:08.154533 2564 patch_prober.go:28] interesting pod/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 22 04:05:08.154758 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:08.154591 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 22 04:05:09.157108 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:09.157065 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:05:09.157604 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:09.157434 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:05:09.158028 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:09.158009 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:05:21.254776 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:21.254746 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46"] Jul 22 04:05:21.255215 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:21.254945 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" containerID="cri-o://e6e60b02597a3948a18f3c7aa30475882a775cedbfef82b77a2d28cdc2163f1a" gracePeriod=30 Jul 22 04:05:24.159573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:24.159536 2564 patch_prober.go:28] interesting pod/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 22 04:05:24.159949 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:24.159606 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:05:26.478356 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.478289 2564 generic.go:356] "Generic (PLEG): container finished" podID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerID="e6e60b02597a3948a18f3c7aa30475882a775cedbfef82b77a2d28cdc2163f1a" exitCode=0 Jul 22 04:05:26.478808 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.478375 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" event={"ID":"38346328-e4d0-4419-9e72-2fc3ba02d461","Type":"ContainerDied","Data":"e6e60b02597a3948a18f3c7aa30475882a775cedbfef82b77a2d28cdc2163f1a"} Jul 22 04:05:26.478808 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.478442 2564 scope.go:117] "RemoveContainer" containerID="d26807065935cf93352801c8f8da7612ef90a49e53b90caba37431c6f5ba7921" Jul 22 04:05:26.509103 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.509086 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:05:26.624312 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624250 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-data\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624312 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624291 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-24l8k\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-kube-api-access-24l8k\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624514 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624322 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-podinfo\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624514 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624410 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38346328-e4d0-4419-9e72-2fc3ba02d461-istiod-ca-cert\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624514 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624441 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-envoy\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624514 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624506 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-certs\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624718 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624537 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-credential-socket\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624718 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624598 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-socket\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624718 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624625 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-token\") pod \"38346328-e4d0-4419-9e72-2fc3ba02d461\" (UID: \"38346328-e4d0-4419-9e72-2fc3ba02d461\") " Jul 22 04:05:26.624946 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624817 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/38346328-e4d0-4419-9e72-2fc3ba02d461-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:05:26.624946 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624899 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:05:26.625284 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.624969 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:05:26.625284 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.625015 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-data" (OuterVolumeSpecName: "istio-data") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:05:26.625432 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.625282 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:05:26.626642 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.626620 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-kube-api-access-24l8k" (OuterVolumeSpecName: "kube-api-access-24l8k") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "kube-api-access-24l8k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:05:26.626848 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.626831 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-token" (OuterVolumeSpecName: "istio-token") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:05:26.627115 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.627094 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:05:26.627278 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.627260 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "38346328-e4d0-4419-9e72-2fc3ba02d461" (UID: "38346328-e4d0-4419-9e72-2fc3ba02d461"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:05:26.725994 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.725973 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.725994 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.725993 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.726111 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.726002 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.726111 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.726011 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-24l8k\" (UniqueName: \"kubernetes.io/projected/38346328-e4d0-4419-9e72-2fc3ba02d461-kube-api-access-24l8k\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.726111 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.726020 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.726111 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.726028 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/38346328-e4d0-4419-9e72-2fc3ba02d461-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.726111 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.726035 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.726111 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.726042 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:26.726111 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:26.726049 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/38346328-e4d0-4419-9e72-2fc3ba02d461-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:05:27.484677 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:27.484646 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" Jul 22 04:05:27.485164 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:27.484648 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46" event={"ID":"38346328-e4d0-4419-9e72-2fc3ba02d461","Type":"ContainerDied","Data":"18d2ebff559d7fd4b8fbb0888cdffae3b58e719380ea77494b24fc018cc9d7ab"} Jul 22 04:05:27.485164 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:27.484786 2564 scope.go:117] "RemoveContainer" containerID="e6e60b02597a3948a18f3c7aa30475882a775cedbfef82b77a2d28cdc2163f1a" Jul 22 04:05:27.509179 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:27.509154 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46"] Jul 22 04:05:27.516868 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:27.516844 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-9cb7aec7-openshift-default-794fcbc7bc-jzn46"] Jul 22 04:05:27.745350 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:27.745305 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" path="/var/lib/kubelet/pods/38346328-e4d0-4419-9e72-2fc3ba02d461/volumes" Jul 22 04:05:41.119361 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119315 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr"] Jul 22 04:05:41.119755 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119697 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" Jul 22 04:05:41.119755 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119708 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" Jul 22 04:05:41.119755 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119716 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" Jul 22 04:05:41.119755 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119722 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" Jul 22 04:05:41.119926 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119781 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" Jul 22 04:05:41.119926 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119790 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="38346328-e4d0-4419-9e72-2fc3ba02d461" containerName="istio-proxy" Jul 22 04:05:41.119926 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.119796 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="400dbe82-5b7d-4c15-bb3d-458caeee8513" containerName="istio-proxy" Jul 22 04:05:41.121961 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.121944 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.126487 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.126460 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2f6647b0-openshift-default-dockercfg-985d7\"" Jul 22 04:05:41.143623 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143599 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-podinfo\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.143759 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143639 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-envoy\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.143759 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143682 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-certs\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.143886 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143757 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-socket\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.143886 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143817 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-data\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.143886 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143870 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istiod-ca-cert\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.144041 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143918 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktvvq\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-kube-api-access-ktvvq\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.144041 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143951 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-credential-socket\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.144041 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143958 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr"] Jul 22 04:05:41.144041 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.143974 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-token\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.244784 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.244748 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-data\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.244992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.244807 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istiod-ca-cert\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.244992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.244854 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktvvq\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-kube-api-access-ktvvq\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.244992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.244880 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-credential-socket\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.244992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.244902 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-token\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.244992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.244928 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-podinfo\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.244992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.244960 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-envoy\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.245285 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.245009 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-certs\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.245285 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.245047 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-socket\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.245388 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.245279 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-data\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.245605 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.245579 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-credential-socket\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.245745 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.245656 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istiod-ca-cert\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.245745 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.245697 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-certs\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.245862 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.245743 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-socket\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.247647 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.247622 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-podinfo\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.247798 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.247783 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-envoy\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.254360 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.254316 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-token\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.254492 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.254473 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktvvq\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-kube-api-access-ktvvq\") pod \"e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.432329 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.432223 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:41.595814 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.595783 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr"] Jul 22 04:05:41.596851 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:05:41.596825 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf79ed9ef_ea2a_4b37_bb03_a4ab9e8c2240.slice/crio-c06aa0dce4188004c2c8780c15916faa858b48587188a9507b70b583ab932fd9 WatchSource:0}: Error finding container c06aa0dce4188004c2c8780c15916faa858b48587188a9507b70b583ab932fd9: Status 404 returned error can't find the container with id c06aa0dce4188004c2c8780c15916faa858b48587188a9507b70b583ab932fd9 Jul 22 04:05:41.598987 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.598956 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:41.599062 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.599018 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:41.599062 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:41.599049 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:42.568283 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:42.568243 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" event={"ID":"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240","Type":"ContainerStarted","Data":"db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8"} Jul 22 04:05:42.568696 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:42.568361 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" event={"ID":"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240","Type":"ContainerStarted","Data":"c06aa0dce4188004c2c8780c15916faa858b48587188a9507b70b583ab932fd9"} Jul 22 04:05:42.591597 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:42.591547 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" podStartSLOduration=1.591534683 podStartE2EDuration="1.591534683s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:05:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:05:42.58995487 +0000 UTC m=+1929.383443361" watchObservedRunningTime="2026-07-22 04:05:42.591534683 +0000 UTC m=+1929.385023175" Jul 22 04:05:43.432718 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:43.432691 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:43.437235 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:43.437198 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:43.573343 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:43.573317 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:43.574067 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:43.574049 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:56.650009 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:56.649943 2564 generic.go:356] "Generic (PLEG): container finished" podID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerID="db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8" exitCode=0 Jul 22 04:05:56.650391 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:56.650019 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" event={"ID":"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240","Type":"ContainerDied","Data":"db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8"} Jul 22 04:05:56.650391 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:56.650385 2564 scope.go:117] "RemoveContainer" containerID="db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8" Jul 22 04:05:56.651316 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:56.651289 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:56.651400 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:56.651384 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:56.651438 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:56.651418 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:05:57.662085 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:57.662049 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" event={"ID":"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240","Type":"ContainerStarted","Data":"452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3"} Jul 22 04:05:58.433098 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:58.433063 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:58.434381 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:58.434353 2564 patch_prober.go:28] interesting pod/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 04:05:58.434517 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:58.434403 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 04:05:58.574098 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:58.574059 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:05:59.432892 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:59.432862 2564 patch_prober.go:28] interesting pod/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 04:05:59.433268 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:05:59.432923 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 04:06:00.432907 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:00.432871 2564 patch_prober.go:28] interesting pod/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 04:06:00.433317 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:00.432926 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 04:06:01.433035 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:01.432993 2564 patch_prober.go:28] interesting pod/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" start-of-body= Jul 22 04:06:01.433549 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:01.433053 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.53:15021/healthz/ready\": dial tcp 10.133.0.53:15021: connect: connection refused" Jul 22 04:06:02.437785 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:02.437737 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:06:02.438849 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:02.438829 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:06:23.066000 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.065967 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw"] Jul 22 04:06:23.068576 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.068561 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" Jul 22 04:06:23.071611 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.071594 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 04:06:23.071723 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.071664 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\"" Jul 22 04:06:23.077156 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.077133 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw"] Jul 22 04:06:23.227933 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.227901 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b9ab84-6006-4421-8d4c-0c57ed8f823d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw\" (UID: \"04b9ab84-6006-4421-8d4c-0c57ed8f823d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" Jul 22 04:06:23.328806 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.328741 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b9ab84-6006-4421-8d4c-0c57ed8f823d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw\" (UID: \"04b9ab84-6006-4421-8d4c-0c57ed8f823d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" Jul 22 04:06:23.329316 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.329299 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b9ab84-6006-4421-8d4c-0c57ed8f823d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw\" (UID: \"04b9ab84-6006-4421-8d4c-0c57ed8f823d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" Jul 22 04:06:23.380381 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.380324 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" Jul 22 04:06:23.522935 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.522912 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw"] Jul 22 04:06:23.524209 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:06:23.524181 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04b9ab84_6006_4421_8d4c_0c57ed8f823d.slice/crio-0a86ae3904ed611558a3e4bd2e289dbb477b75ce188393537aa5827957c57352 WatchSource:0}: Error finding container 0a86ae3904ed611558a3e4bd2e289dbb477b75ce188393537aa5827957c57352: Status 404 returned error can't find the container with id 0a86ae3904ed611558a3e4bd2e289dbb477b75ce188393537aa5827957c57352 Jul 22 04:06:23.806807 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.806776 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" event={"ID":"04b9ab84-6006-4421-8d4c-0c57ed8f823d","Type":"ContainerStarted","Data":"0a48ee028172e208639a2423bfa561a4b59c4f64b339ed3ccc33faa498c05ddb"} Jul 22 04:06:23.806927 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.806835 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" event={"ID":"04b9ab84-6006-4421-8d4c-0c57ed8f823d","Type":"ContainerStarted","Data":"0a86ae3904ed611558a3e4bd2e289dbb477b75ce188393537aa5827957c57352"} Jul 22 04:06:23.826699 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:23.826659 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" podStartSLOduration=0.826643281 podStartE2EDuration="826.643281ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:06:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:06:23.824994027 +0000 UTC m=+1970.618482520" watchObservedRunningTime="2026-07-22 04:06:23.826643281 +0000 UTC m=+1970.620131772" Jul 22 04:06:24.813458 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:24.813429 2564 generic.go:356] "Generic (PLEG): container finished" podID="04b9ab84-6006-4421-8d4c-0c57ed8f823d" containerID="0a48ee028172e208639a2423bfa561a4b59c4f64b339ed3ccc33faa498c05ddb" exitCode=0 Jul 22 04:06:24.813806 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:24.813518 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" event={"ID":"04b9ab84-6006-4421-8d4c-0c57ed8f823d","Type":"ContainerDied","Data":"0a48ee028172e208639a2423bfa561a4b59c4f64b339ed3ccc33faa498c05ddb"} Jul 22 04:06:25.948275 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:25.948255 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" Jul 22 04:06:26.046083 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:26.046053 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b9ab84-6006-4421-8d4c-0c57ed8f823d-maas-api-service-ca\") pod \"04b9ab84-6006-4421-8d4c-0c57ed8f823d\" (UID: \"04b9ab84-6006-4421-8d4c-0c57ed8f823d\") " Jul 22 04:06:26.046417 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:26.046396 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04b9ab84-6006-4421-8d4c-0c57ed8f823d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "04b9ab84-6006-4421-8d4c-0c57ed8f823d" (UID: "04b9ab84-6006-4421-8d4c-0c57ed8f823d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:06:26.147668 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:26.147606 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/04b9ab84-6006-4421-8d4c-0c57ed8f823d-maas-api-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:06:26.823717 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:26.823692 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" Jul 22 04:06:26.823858 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:26.823740 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw" event={"ID":"04b9ab84-6006-4421-8d4c-0c57ed8f823d","Type":"ContainerDied","Data":"0a86ae3904ed611558a3e4bd2e289dbb477b75ce188393537aa5827957c57352"} Jul 22 04:06:26.823858 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:26.823782 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0a86ae3904ed611558a3e4bd2e289dbb477b75ce188393537aa5827957c57352" Jul 22 04:06:33.869588 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:33.869557 2564 scope.go:117] "RemoveContainer" containerID="c7686205ff57b2f226632641025e9df77266dca368fc6a808efde5362d246229" Jul 22 04:06:43.167704 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:43.167665 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw"] Jul 22 04:06:43.170362 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:43.170306 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2f6647b0-9ljmw"] Jul 22 04:06:43.745911 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:06:43.745875 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04b9ab84-6006-4421-8d4c-0c57ed8f823d" path="/var/lib/kubelet/pods/04b9ab84-6006-4421-8d4c-0c57ed8f823d/volumes" Jul 22 04:07:33.599562 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:33.599530 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr"] Jul 22 04:07:33.600110 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:33.599715 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" containerID="cri-o://452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3" gracePeriod=30 Jul 22 04:07:33.890382 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:33.890314 2564 scope.go:117] "RemoveContainer" containerID="3f48af870155160701dc50ae2e451a296c473512ab468a0164c69d85b86faee3" Jul 22 04:07:38.874032 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.874010 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:07:38.957061 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957039 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istiod-ca-cert\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957168 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957070 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ktvvq\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-kube-api-access-ktvvq\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957168 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957100 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-envoy\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957168 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957132 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-socket\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957168 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957162 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-certs\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957378 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957202 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-data\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957378 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957247 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-podinfo\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957378 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957273 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-credential-socket\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957378 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957300 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-token\") pod \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\" (UID: \"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240\") " Jul 22 04:07:38.957582 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957470 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:07:38.957582 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957493 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:07:38.957582 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957530 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:07:38.957714 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957645 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:38.957714 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957650 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:07:38.957714 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957663 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:38.957714 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.957687 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:38.958183 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.958161 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-data" (OuterVolumeSpecName: "istio-data") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:07:38.959269 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.959243 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-kube-api-access-ktvvq" (OuterVolumeSpecName: "kube-api-access-ktvvq") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "kube-api-access-ktvvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:07:38.959514 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.959494 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:07:38.959945 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.959922 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:07:38.960024 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:38.959954 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-token" (OuterVolumeSpecName: "istio-token") pod "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" (UID: "f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:07:39.058832 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.058808 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ktvvq\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-kube-api-access-ktvvq\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:39.058832 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.058833 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:39.058965 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.058846 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:39.058965 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.058857 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:39.058965 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.058868 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:39.058965 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.058881 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:07:39.206104 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.206052 2564 generic.go:356] "Generic (PLEG): container finished" podID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerID="452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3" exitCode=0 Jul 22 04:07:39.206104 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.206102 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" Jul 22 04:07:39.206250 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.206134 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" event={"ID":"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240","Type":"ContainerDied","Data":"452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3"} Jul 22 04:07:39.206250 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.206197 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr" event={"ID":"f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240","Type":"ContainerDied","Data":"c06aa0dce4188004c2c8780c15916faa858b48587188a9507b70b583ab932fd9"} Jul 22 04:07:39.206250 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.206230 2564 scope.go:117] "RemoveContainer" containerID="452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3" Jul 22 04:07:39.214919 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.214904 2564 scope.go:117] "RemoveContainer" containerID="db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8" Jul 22 04:07:39.222990 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.222967 2564 scope.go:117] "RemoveContainer" containerID="452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3" Jul 22 04:07:39.223237 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:07:39.223217 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3\": container with ID starting with 452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3 not found: ID does not exist" containerID="452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3" Jul 22 04:07:39.223308 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.223244 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3"} err="failed to get container status \"452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3\": rpc error: code = NotFound desc = could not find container \"452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3\": container with ID starting with 452e286a804c14e8002faacfb8eaf8b6e8dfd2ab9dee05f0b58be19d597a25a3 not found: ID does not exist" Jul 22 04:07:39.223308 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.223260 2564 scope.go:117] "RemoveContainer" containerID="db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8" Jul 22 04:07:39.223513 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:07:39.223497 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8\": container with ID starting with db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8 not found: ID does not exist" containerID="db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8" Jul 22 04:07:39.223552 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.223519 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8"} err="failed to get container status \"db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8\": rpc error: code = NotFound desc = could not find container \"db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8\": container with ID starting with db8797d92cd161808c2db355bab41447ac9bee99a69ba0ea1b8b2b29f0a8eeb8 not found: ID does not exist" Jul 22 04:07:39.232190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.232165 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr"] Jul 22 04:07:39.238502 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.238480 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2f6647b0-openshift-default-f66db846c-txrgr"] Jul 22 04:07:39.745410 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:07:39.745387 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" path="/var/lib/kubelet/pods/f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240/volumes" Jul 22 04:08:30.448976 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.448942 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll"] Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449269 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449280 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449295 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449302 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449317 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04b9ab84-6006-4421-8d4c-0c57ed8f823d" containerName="revoke-keys" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449322 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="04b9ab84-6006-4421-8d4c-0c57ed8f823d" containerName="revoke-keys" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449407 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449418 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="f79ed9ef-ea2a-4b37-bb03-a4ab9e8c2240" containerName="istio-proxy" Jul 22 04:08:30.449441 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.449424 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="04b9ab84-6006-4421-8d4c-0c57ed8f823d" containerName="revoke-keys" Jul 22 04:08:30.452436 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.452420 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.456102 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.456079 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-41e8d3-openshift-default-dockercfg-zpttw\"" Jul 22 04:08:30.470029 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.470006 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll"] Jul 22 04:08:30.545514 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545482 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-istio-token\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545694 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545531 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-data\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545694 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545607 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-certs\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545694 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545667 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-socket\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545857 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545721 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84de7f82-b554-49c4-abb3-e620d13161e9-istiod-ca-cert\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545857 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545772 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-envoy\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545857 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545809 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-credential-socket\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545857 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84de7f82-b554-49c4-abb3-e620d13161e9-istio-podinfo\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.545988 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.545871 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bdfd\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-kube-api-access-8bdfd\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647360 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647304 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-socket\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647515 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647368 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84de7f82-b554-49c4-abb3-e620d13161e9-istiod-ca-cert\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647515 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647392 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-envoy\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647515 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647409 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-credential-socket\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647515 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647430 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84de7f82-b554-49c4-abb3-e620d13161e9-istio-podinfo\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647515 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647449 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bdfd\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-kube-api-access-8bdfd\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647515 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647492 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-istio-token\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647825 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647522 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-data\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647825 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647550 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-certs\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.647825 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.647732 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-socket\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.648071 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.648045 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-credential-socket\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.648202 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.648113 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-data\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.648297 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.648178 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-certs\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.648786 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.648767 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84de7f82-b554-49c4-abb3-e620d13161e9-istiod-ca-cert\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.650027 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.649999 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-envoy\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.650203 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.650186 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84de7f82-b554-49c4-abb3-e620d13161e9-istio-podinfo\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.656225 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.656200 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-istio-token\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.656313 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.656291 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bdfd\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-kube-api-access-8bdfd\") pod \"e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.762464 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.762445 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:30.900619 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.900591 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll"] Jul 22 04:08:30.901663 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:08:30.901637 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84de7f82_b554_49c4_abb3_e620d13161e9.slice/crio-eb852e84b43f301a55a5026d104502381c93f6bb15f010727cb2b67e84a183c9 WatchSource:0}: Error finding container eb852e84b43f301a55a5026d104502381c93f6bb15f010727cb2b67e84a183c9: Status 404 returned error can't find the container with id eb852e84b43f301a55a5026d104502381c93f6bb15f010727cb2b67e84a183c9 Jul 22 04:08:30.904281 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.904246 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:30.904406 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.904321 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:30.904406 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:30.904388 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:31.478606 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:31.478570 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" event={"ID":"84de7f82-b554-49c4-abb3-e620d13161e9","Type":"ContainerStarted","Data":"ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7"} Jul 22 04:08:31.479064 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:31.478630 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" event={"ID":"84de7f82-b554-49c4-abb3-e620d13161e9","Type":"ContainerStarted","Data":"eb852e84b43f301a55a5026d104502381c93f6bb15f010727cb2b67e84a183c9"} Jul 22 04:08:31.504736 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:31.504683 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" podStartSLOduration=1.504663414 podStartE2EDuration="1.504663414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:08:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:08:31.501249918 +0000 UTC m=+2098.294738421" watchObservedRunningTime="2026-07-22 04:08:31.504663414 +0000 UTC m=+2098.298151907" Jul 22 04:08:31.763357 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:31.763301 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:31.767823 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:31.767800 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:32.483775 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:32.483750 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:32.484604 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:32.484587 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:38.556045 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.556004 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427"] Jul 22 04:08:38.559864 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.559841 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.563169 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.563151 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-b9a16f-openshift-default-dockercfg-fw2r6\"" Jul 22 04:08:38.594912 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.594866 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427"] Jul 22 04:08:38.715781 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.715731 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-credential-socket\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.715781 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.715780 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-certs\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.716049 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.715842 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-podinfo\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.716049 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.715895 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-socket\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.716049 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.715945 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04ac0787-0382-45ca-9604-12dacb0b40dd-istiod-ca-cert\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.716049 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.715985 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-envoy\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.716049 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.716044 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snjns\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-kube-api-access-snjns\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.716253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.716072 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-data\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.716253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.716113 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-token\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817405 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817309 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-podinfo\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817405 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817396 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-socket\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817417 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04ac0787-0382-45ca-9604-12dacb0b40dd-istiod-ca-cert\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817436 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-envoy\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817470 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snjns\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-kube-api-access-snjns\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817489 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-data\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817520 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-token\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817593 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-credential-socket\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.817628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.817614 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-certs\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.818032 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.818004 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-certs\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.818103 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.818067 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-credential-socket\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.818203 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.818176 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-data\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.818510 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.818205 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-socket\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.818510 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.818240 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04ac0787-0382-45ca-9604-12dacb0b40dd-istiod-ca-cert\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.819892 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.819872 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-envoy\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.820096 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.820080 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-podinfo\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.826784 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.826758 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snjns\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-kube-api-access-snjns\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.826894 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.826874 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-token\") pod \"e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:38.870103 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:38.870085 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:39.054042 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.054015 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427"] Jul 22 04:08:39.055177 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:08:39.055150 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04ac0787_0382_45ca_9604_12dacb0b40dd.slice/crio-2aa89368f38df9b7520e27965ca42b1f27213c3ab6b45737b16e3a5ba0d3bb94 WatchSource:0}: Error finding container 2aa89368f38df9b7520e27965ca42b1f27213c3ab6b45737b16e3a5ba0d3bb94: Status 404 returned error can't find the container with id 2aa89368f38df9b7520e27965ca42b1f27213c3ab6b45737b16e3a5ba0d3bb94 Jul 22 04:08:39.057145 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.057116 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:39.057220 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.057181 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:39.057260 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.057217 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:39.521533 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.521495 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" event={"ID":"04ac0787-0382-45ca-9604-12dacb0b40dd","Type":"ContainerStarted","Data":"7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523"} Jul 22 04:08:39.521710 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.521561 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" event={"ID":"04ac0787-0382-45ca-9604-12dacb0b40dd","Type":"ContainerStarted","Data":"2aa89368f38df9b7520e27965ca42b1f27213c3ab6b45737b16e3a5ba0d3bb94"} Jul 22 04:08:39.554722 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.554670 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" podStartSLOduration=1.554656078 podStartE2EDuration="1.554656078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:08:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:08:39.551472964 +0000 UTC m=+2106.344961454" watchObservedRunningTime="2026-07-22 04:08:39.554656078 +0000 UTC m=+2106.348144569" Jul 22 04:08:39.870870 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:39.870793 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:40.875608 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:40.875581 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:41.532050 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:41.532022 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:41.532906 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:41.532886 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:45.560585 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:45.560547 2564 generic.go:356] "Generic (PLEG): container finished" podID="84de7f82-b554-49c4-abb3-e620d13161e9" containerID="ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7" exitCode=0 Jul 22 04:08:45.560985 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:45.560597 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" event={"ID":"84de7f82-b554-49c4-abb3-e620d13161e9","Type":"ContainerDied","Data":"ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7"} Jul 22 04:08:45.560985 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:45.560980 2564 scope.go:117] "RemoveContainer" containerID="ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7" Jul 22 04:08:45.562021 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:45.561985 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:45.562141 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:45.562074 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:45.562214 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:45.562140 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:46.568492 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:46.568461 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" event={"ID":"84de7f82-b554-49c4-abb3-e620d13161e9","Type":"ContainerStarted","Data":"b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5"} Jul 22 04:08:46.763473 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:46.763437 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:46.764428 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:46.764407 2564 patch_prober.go:28] interesting pod/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 04:08:46.764486 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:46.764457 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 04:08:47.484466 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:47.484432 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:47.762944 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:47.762911 2564 patch_prober.go:28] interesting pod/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 04:08:47.763319 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:47.762965 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 04:08:48.762836 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:48.762799 2564 patch_prober.go:28] interesting pod/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 04:08:48.763008 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:48.762865 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 04:08:49.762818 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:49.762789 2564 patch_prober.go:28] interesting pod/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 22 04:08:49.762966 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:49.762847 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 22 04:08:50.767244 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:50.767209 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:50.768454 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:50.768429 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:08:52.605358 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:52.605290 2564 generic.go:356] "Generic (PLEG): container finished" podID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerID="7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523" exitCode=0 Jul 22 04:08:52.605768 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:52.605372 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" event={"ID":"04ac0787-0382-45ca-9604-12dacb0b40dd","Type":"ContainerDied","Data":"7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523"} Jul 22 04:08:52.605859 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:52.605844 2564 scope.go:117] "RemoveContainer" containerID="7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523" Jul 22 04:08:52.606825 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:52.606788 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:52.606922 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:52.606851 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:52.606922 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:52.606883 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:08:53.613117 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:53.613085 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" event={"ID":"04ac0787-0382-45ca-9604-12dacb0b40dd","Type":"ContainerStarted","Data":"37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac"} Jul 22 04:08:53.870917 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:53.870835 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:53.872013 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:53.871987 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 04:08:53.872105 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:53.872049 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 04:08:54.871149 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:54.871114 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 04:08:54.871557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:54.871177 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 04:08:55.871037 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:55.871003 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 04:08:55.871206 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:55.871058 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 04:08:56.532821 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:56.532791 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:56.870965 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:56.870883 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 22 04:08:56.870965 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:56.870938 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 22 04:08:57.873645 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:57.873613 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:08:57.874635 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:08:57.874616 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:10:12.210080 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.210042 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq"] Jul 22 04:10:12.212578 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.212556 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" Jul 22 04:10:12.215398 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.215379 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 04:10:12.215491 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.215441 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\"" Jul 22 04:10:12.221607 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.221585 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq"] Jul 22 04:10:12.247271 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.247243 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17cb33e4-111b-4ec8-8fa5-76cfc74d38b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq\" (UID: \"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" Jul 22 04:10:12.348689 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.348664 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17cb33e4-111b-4ec8-8fa5-76cfc74d38b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq\" (UID: \"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" Jul 22 04:10:12.349282 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.349264 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17cb33e4-111b-4ec8-8fa5-76cfc74d38b4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq\" (UID: \"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" Jul 22 04:10:12.524808 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.524787 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" Jul 22 04:10:12.653549 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:12.653521 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq"] Jul 22 04:10:12.654389 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:10:12.654364 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17cb33e4_111b_4ec8_8fa5_76cfc74d38b4.slice/crio-ec494e93bdf73335a0bcab8e4f379fd6833521a9237d2c95172fb9aed62a4c53 WatchSource:0}: Error finding container ec494e93bdf73335a0bcab8e4f379fd6833521a9237d2c95172fb9aed62a4c53: Status 404 returned error can't find the container with id ec494e93bdf73335a0bcab8e4f379fd6833521a9237d2c95172fb9aed62a4c53 Jul 22 04:10:13.017232 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:13.017200 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" event={"ID":"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4","Type":"ContainerStarted","Data":"fbf5c0ad9ddf0cc782d8e14b6293d1f244f2833dd313a0cf1a386c96cc97488c"} Jul 22 04:10:13.017383 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:13.017257 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" event={"ID":"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4","Type":"ContainerStarted","Data":"ec494e93bdf73335a0bcab8e4f379fd6833521a9237d2c95172fb9aed62a4c53"} Jul 22 04:10:13.033589 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:13.033543 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" podStartSLOduration=1.033528459 podStartE2EDuration="1.033528459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:10:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:10:13.032771663 +0000 UTC m=+2199.826260155" watchObservedRunningTime="2026-07-22 04:10:13.033528459 +0000 UTC m=+2199.827016948" Jul 22 04:10:14.023887 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:14.023857 2564 generic.go:356] "Generic (PLEG): container finished" podID="17cb33e4-111b-4ec8-8fa5-76cfc74d38b4" containerID="fbf5c0ad9ddf0cc782d8e14b6293d1f244f2833dd313a0cf1a386c96cc97488c" exitCode=0 Jul 22 04:10:14.024226 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:14.023938 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" event={"ID":"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4","Type":"ContainerDied","Data":"fbf5c0ad9ddf0cc782d8e14b6293d1f244f2833dd313a0cf1a386c96cc97488c"} Jul 22 04:10:15.155888 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:15.155864 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" Jul 22 04:10:15.171445 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:15.171409 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17cb33e4-111b-4ec8-8fa5-76cfc74d38b4-maas-api-service-ca\") pod \"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4\" (UID: \"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4\") " Jul 22 04:10:15.171788 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:15.171751 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17cb33e4-111b-4ec8-8fa5-76cfc74d38b4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "17cb33e4-111b-4ec8-8fa5-76cfc74d38b4" (UID: "17cb33e4-111b-4ec8-8fa5-76cfc74d38b4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:10:15.171908 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:15.171859 2564 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/17cb33e4-111b-4ec8-8fa5-76cfc74d38b4-maas-api-service-ca\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:16.035630 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:16.035550 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" Jul 22 04:10:16.035630 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:16.035587 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq" event={"ID":"17cb33e4-111b-4ec8-8fa5-76cfc74d38b4","Type":"ContainerDied","Data":"ec494e93bdf73335a0bcab8e4f379fd6833521a9237d2c95172fb9aed62a4c53"} Jul 22 04:10:16.035809 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:16.035635 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ec494e93bdf73335a0bcab8e4f379fd6833521a9237d2c95172fb9aed62a4c53" Jul 22 04:10:27.300356 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:27.300308 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq"] Jul 22 04:10:27.304749 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:27.304725 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-41e8d3-zskwq"] Jul 22 04:10:27.744738 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:27.744709 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17cb33e4-111b-4ec8-8fa5-76cfc74d38b4" path="/var/lib/kubelet/pods/17cb33e4-111b-4ec8-8fa5-76cfc74d38b4/volumes" Jul 22 04:10:34.117433 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:34.117403 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll"] Jul 22 04:10:34.117911 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:34.117583 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" containerID="cri-o://b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5" gracePeriod=30 Jul 22 04:10:35.769467 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:35.769428 2564 patch_prober.go:28] interesting pod/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 04:10:35.769837 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:35.769496 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:10:39.371346 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.371315 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:10:39.462399 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462320 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bdfd\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-kube-api-access-8bdfd\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462399 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462391 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-socket\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462584 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462413 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-certs\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462584 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462432 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-istio-token\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462584 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462451 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-credential-socket\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462584 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462466 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84de7f82-b554-49c4-abb3-e620d13161e9-istio-podinfo\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462584 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462500 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84de7f82-b554-49c4-abb3-e620d13161e9-istiod-ca-cert\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462584 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462531 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-data\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462584 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462558 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-envoy\") pod \"84de7f82-b554-49c4-abb3-e620d13161e9\" (UID: \"84de7f82-b554-49c4-abb3-e620d13161e9\") " Jul 22 04:10:39.462937 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462731 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:10:39.462937 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462748 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:10:39.462937 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462812 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:10:39.462937 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462870 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.462937 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.462889 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.463381 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.463051 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84de7f82-b554-49c4-abb3-e620d13161e9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:10:39.463821 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.463792 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-data" (OuterVolumeSpecName: "istio-data") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:10:39.464686 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.464656 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84de7f82-b554-49c4-abb3-e620d13161e9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:10:39.464686 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.464668 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-kube-api-access-8bdfd" (OuterVolumeSpecName: "kube-api-access-8bdfd") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "kube-api-access-8bdfd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:10:39.464812 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.464716 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-istio-token" (OuterVolumeSpecName: "istio-token") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:10:39.464933 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.464919 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "84de7f82-b554-49c4-abb3-e620d13161e9" (UID: "84de7f82-b554-49c4-abb3-e620d13161e9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:10:39.563223 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.563200 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bdfd\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-kube-api-access-8bdfd\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.563223 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.563221 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.563371 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.563230 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84de7f82-b554-49c4-abb3-e620d13161e9-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.563371 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.563238 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84de7f82-b554-49c4-abb3-e620d13161e9-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.563371 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.563245 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84de7f82-b554-49c4-abb3-e620d13161e9-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.563371 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.563253 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:39.563371 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:39.563260 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84de7f82-b554-49c4-abb3-e620d13161e9-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:10:40.155774 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.155715 2564 generic.go:356] "Generic (PLEG): container finished" podID="84de7f82-b554-49c4-abb3-e620d13161e9" containerID="b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5" exitCode=0 Jul 22 04:10:40.155774 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.155760 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" Jul 22 04:10:40.155936 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.155796 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" event={"ID":"84de7f82-b554-49c4-abb3-e620d13161e9","Type":"ContainerDied","Data":"b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5"} Jul 22 04:10:40.155936 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.155850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll" event={"ID":"84de7f82-b554-49c4-abb3-e620d13161e9","Type":"ContainerDied","Data":"eb852e84b43f301a55a5026d104502381c93f6bb15f010727cb2b67e84a183c9"} Jul 22 04:10:40.155936 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.155879 2564 scope.go:117] "RemoveContainer" containerID="b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5" Jul 22 04:10:40.165204 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.165186 2564 scope.go:117] "RemoveContainer" containerID="ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7" Jul 22 04:10:40.173133 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.173095 2564 scope.go:117] "RemoveContainer" containerID="b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5" Jul 22 04:10:40.173499 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:10:40.173449 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5\": container with ID starting with b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5 not found: ID does not exist" containerID="b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5" Jul 22 04:10:40.173499 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.173483 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5"} err="failed to get container status \"b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5\": rpc error: code = NotFound desc = could not find container \"b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5\": container with ID starting with b67f4bd1221c0b7b1c5c0be7d8cd734624d15efa124b82095a6918ec8a6c06d5 not found: ID does not exist" Jul 22 04:10:40.173682 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.173505 2564 scope.go:117] "RemoveContainer" containerID="ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7" Jul 22 04:10:40.174042 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:10:40.174022 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7\": container with ID starting with ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7 not found: ID does not exist" containerID="ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7" Jul 22 04:10:40.174132 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.174046 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7"} err="failed to get container status \"ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7\": rpc error: code = NotFound desc = could not find container \"ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7\": container with ID starting with ca35ad860eed9fd99d7328355b1c304c87ea4cb396c531e69941ef0972fe7bd7 not found: ID does not exist" Jul 22 04:10:40.176314 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.176290 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll"] Jul 22 04:10:40.180205 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:40.180186 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-41e8d3-openshift-default-74f8f5475-ww2ll"] Jul 22 04:10:41.745590 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:41.745562 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" path="/var/lib/kubelet/pods/84de7f82-b554-49c4-abb3-e620d13161e9/volumes" Jul 22 04:10:56.007122 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.007079 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427"] Jul 22 04:10:56.007546 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.007378 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" containerID="cri-o://37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac" gracePeriod=30 Jul 22 04:10:56.914707 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.914675 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz"] Jul 22 04:10:56.915176 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915151 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17cb33e4-111b-4ec8-8fa5-76cfc74d38b4" containerName="revoke-keys" Jul 22 04:10:56.915176 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915178 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="17cb33e4-111b-4ec8-8fa5-76cfc74d38b4" containerName="revoke-keys" Jul 22 04:10:56.915347 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915195 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" Jul 22 04:10:56.915347 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915204 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" Jul 22 04:10:56.915347 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915212 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" Jul 22 04:10:56.915347 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915221 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" Jul 22 04:10:56.915477 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915330 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" Jul 22 04:10:56.915477 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915364 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="84de7f82-b554-49c4-abb3-e620d13161e9" containerName="istio-proxy" Jul 22 04:10:56.915477 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.915373 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="17cb33e4-111b-4ec8-8fa5-76cfc74d38b4" containerName="revoke-keys" Jul 22 04:10:56.918973 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.918957 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.922200 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.922179 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-b721b6-openshift-default-dockercfg-s7x7b\"" Jul 22 04:10:56.930807 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.930785 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz"] Jul 22 04:10:56.993193 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993161 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-data\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993373 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993203 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-certs\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993373 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993232 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istiod-ca-cert\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993373 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993261 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-podinfo\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993373 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993286 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-socket\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993435 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-token\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993484 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-envoy\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993516 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-credential-socket\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:56.993557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:56.993545 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgspc\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-kube-api-access-mgspc\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.094904 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.094871 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-token\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.094910 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-envoy\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.094930 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-credential-socket\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.094946 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgspc\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-kube-api-access-mgspc\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.094997 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-data\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095020 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-certs\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095047 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istiod-ca-cert\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095075 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-podinfo\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-socket\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.095380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095370 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-credential-socket\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.096016 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095520 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-data\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.096016 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095609 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-certs\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.096016 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095647 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-socket\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.096016 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.095822 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istiod-ca-cert\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.097562 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.097532 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-envoy\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.097853 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.097831 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-podinfo\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.103187 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.103165 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-token\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.103422 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.103407 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgspc\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-kube-api-access-mgspc\") pod \"e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.230419 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.230365 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:57.367434 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.367407 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz"] Jul 22 04:10:57.369142 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:10:57.369120 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a7addfe_8cf0_4ddd_a2b4_41388281bcb1.slice/crio-9dd5ffdf62b141b40cafe180b2c356833fee9894d985c9692cc638947e2d5a46 WatchSource:0}: Error finding container 9dd5ffdf62b141b40cafe180b2c356833fee9894d985c9692cc638947e2d5a46: Status 404 returned error can't find the container with id 9dd5ffdf62b141b40cafe180b2c356833fee9894d985c9692cc638947e2d5a46 Jul 22 04:10:57.371566 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.371534 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:10:57.371643 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.371616 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:10:57.371684 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.371648 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:10:57.876460 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.876421 2564 patch_prober.go:28] interesting pod/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 04:10:57.876624 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:57.876483 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:10:58.251280 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:58.251252 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" event={"ID":"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1","Type":"ContainerStarted","Data":"5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c"} Jul 22 04:10:58.251649 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:58.251301 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" event={"ID":"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1","Type":"ContainerStarted","Data":"9dd5ffdf62b141b40cafe180b2c356833fee9894d985c9692cc638947e2d5a46"} Jul 22 04:10:58.273445 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:58.273400 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podStartSLOduration=2.273363942 podStartE2EDuration="2.273363942s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:10:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:10:58.272456751 +0000 UTC m=+2245.065945242" watchObservedRunningTime="2026-07-22 04:10:58.273363942 +0000 UTC m=+2245.066852429" Jul 22 04:10:59.231256 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:59.231224 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:59.236311 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:59.236288 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:59.257075 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:59.257056 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:10:59.258055 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:10:59.257860 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:11:01.260046 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.260021 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:11:01.269248 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.269188 2564 generic.go:356] "Generic (PLEG): container finished" podID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerID="37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac" exitCode=0 Jul 22 04:11:01.269248 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.269239 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" Jul 22 04:11:01.269459 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.269245 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" event={"ID":"04ac0787-0382-45ca-9604-12dacb0b40dd","Type":"ContainerDied","Data":"37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac"} Jul 22 04:11:01.269459 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.269311 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427" event={"ID":"04ac0787-0382-45ca-9604-12dacb0b40dd","Type":"ContainerDied","Data":"2aa89368f38df9b7520e27965ca42b1f27213c3ab6b45737b16e3a5ba0d3bb94"} Jul 22 04:11:01.269459 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.269377 2564 scope.go:117] "RemoveContainer" containerID="37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac" Jul 22 04:11:01.278227 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.278209 2564 scope.go:117] "RemoveContainer" containerID="7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523" Jul 22 04:11:01.288820 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.288801 2564 scope.go:117] "RemoveContainer" containerID="37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac" Jul 22 04:11:01.289067 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:11:01.289051 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac\": container with ID starting with 37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac not found: ID does not exist" containerID="37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac" Jul 22 04:11:01.289122 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.289085 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac"} err="failed to get container status \"37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac\": rpc error: code = NotFound desc = could not find container \"37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac\": container with ID starting with 37078a57674fc41ddabaf0560191f56f6c251eda88b9ccf0fb4cdde9822f57ac not found: ID does not exist" Jul 22 04:11:01.289122 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.289101 2564 scope.go:117] "RemoveContainer" containerID="7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523" Jul 22 04:11:01.289390 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:11:01.289373 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523\": container with ID starting with 7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523 not found: ID does not exist" containerID="7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523" Jul 22 04:11:01.289443 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.289394 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523"} err="failed to get container status \"7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523\": rpc error: code = NotFound desc = could not find container \"7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523\": container with ID starting with 7f403336753c7539ecae58417ae4723ded85e81694a5992b1e5e35fcdace1523 not found: ID does not exist" Jul 22 04:11:01.431005 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.430973 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-socket\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431170 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431024 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-podinfo\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431170 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431050 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-envoy\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431170 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431070 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-certs\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431170 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431092 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-credential-socket\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431170 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431118 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04ac0787-0382-45ca-9604-12dacb0b40dd-istiod-ca-cert\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431462 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431236 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-data\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431462 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431290 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-token\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431462 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431300 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:01.431462 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431404 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-snjns\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-kube-api-access-snjns\") pod \"04ac0787-0382-45ca-9604-12dacb0b40dd\" (UID: \"04ac0787-0382-45ca-9604-12dacb0b40dd\") " Jul 22 04:11:01.431672 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431461 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:01.431672 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431584 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04ac0787-0382-45ca-9604-12dacb0b40dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:11:01.431887 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431664 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:01.431950 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431912 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.432485 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.431932 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.432485 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.432030 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/04ac0787-0382-45ca-9604-12dacb0b40dd-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.432485 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.432120 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.432677 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.432570 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:01.433529 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.433507 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:11:01.433642 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.433624 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:11:01.433835 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.433811 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:01.434127 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.434109 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-kube-api-access-snjns" (OuterVolumeSpecName: "kube-api-access-snjns") pod "04ac0787-0382-45ca-9604-12dacb0b40dd" (UID: "04ac0787-0382-45ca-9604-12dacb0b40dd"). InnerVolumeSpecName "kube-api-access-snjns". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:11:01.532559 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.532502 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snjns\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-kube-api-access-snjns\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.532559 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.532525 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.532559 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.532534 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.532559 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.532541 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.532559 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.532548 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/04ac0787-0382-45ca-9604-12dacb0b40dd-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:01.592364 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.592319 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427"] Jul 22 04:11:01.597219 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.597199 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-b9a16f-openshift-default-56955bdbdf-6v427"] Jul 22 04:11:01.744752 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:01.744723 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" path="/var/lib/kubelet/pods/04ac0787-0382-45ca-9604-12dacb0b40dd/volumes" Jul 22 04:11:05.466931 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.466894 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72"] Jul 22 04:11:05.467347 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.467315 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" Jul 22 04:11:05.467402 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.467371 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" Jul 22 04:11:05.467525 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.467512 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" Jul 22 04:11:05.467568 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.467528 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" Jul 22 04:11:05.470610 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.470591 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.473733 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.473711 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-656735-openshift-default-dockercfg-hvjfd\"" Jul 22 04:11:05.483156 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.483131 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72"] Jul 22 04:11:05.566995 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.566951 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkjkv\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-kube-api-access-fkjkv\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567183 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567007 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-podinfo\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567183 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567055 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-token\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567183 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567100 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istiod-ca-cert\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567183 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567158 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-data\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567221 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-credential-socket\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567271 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-certs\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567328 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-envoy\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.567573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.567389 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-socket\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668315 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668282 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-certs\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668500 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668325 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-envoy\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668500 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668380 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-socket\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668500 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668416 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkjkv\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-kube-api-access-fkjkv\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668500 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668442 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-podinfo\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668500 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668482 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-token\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668707 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668514 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istiod-ca-cert\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668707 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668563 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-data\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668707 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668600 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-credential-socket\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668853 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668719 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-certs\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668853 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668786 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-socket\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.668959 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.668929 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-credential-socket\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.669064 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.669041 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-data\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.669302 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.669285 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istiod-ca-cert\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.670799 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.670778 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-envoy\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.671243 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.671224 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-podinfo\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.678887 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.678840 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkjkv\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-kube-api-access-fkjkv\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.679207 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.679187 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-token\") pod \"e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.782199 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.782173 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:05.925711 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.925671 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72"] Jul 22 04:11:05.927287 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:11:05.927258 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56676aba_bcd7_4dfa_98c7_bb1f5f7e1084.slice/crio-49315a21e66b2fba5ab9818f7723894a4c4ca7f6078c8080522d5a6d3660e0b4 WatchSource:0}: Error finding container 49315a21e66b2fba5ab9818f7723894a4c4ca7f6078c8080522d5a6d3660e0b4: Status 404 returned error can't find the container with id 49315a21e66b2fba5ab9818f7723894a4c4ca7f6078c8080522d5a6d3660e0b4 Jul 22 04:11:05.929423 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.929386 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:05.929489 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.929468 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:05.929531 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:05.929505 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:06.299761 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:06.299725 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" event={"ID":"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084","Type":"ContainerStarted","Data":"47b5729ceecfa3d03f50136ae5a37de6d8d5fa3b96e7f31a3720d716338f4349"} Jul 22 04:11:06.299926 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:06.299792 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" event={"ID":"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084","Type":"ContainerStarted","Data":"49315a21e66b2fba5ab9818f7723894a4c4ca7f6078c8080522d5a6d3660e0b4"} Jul 22 04:11:06.323221 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:06.323169 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" podStartSLOduration=1.323156425 podStartE2EDuration="1.323156425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:11:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:11:06.320754117 +0000 UTC m=+2253.114242609" watchObservedRunningTime="2026-07-22 04:11:06.323156425 +0000 UTC m=+2253.116644915" Jul 22 04:11:06.782659 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:06.782634 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:06.787490 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:06.787471 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:07.305417 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:07.305394 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:07.306191 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:07.306170 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:15.299592 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:15.299539 2564 patch_prober.go:28] interesting pod/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 04:11:15.300616 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:15.300252 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 04:11:16.359240 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:16.359203 2564 generic.go:356] "Generic (PLEG): container finished" podID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerID="5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c" exitCode=0 Jul 22 04:11:16.359690 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:16.359277 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" event={"ID":"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1","Type":"ContainerDied","Data":"5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c"} Jul 22 04:11:16.359733 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:16.359716 2564 scope.go:117] "RemoveContainer" containerID="5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c" Jul 22 04:11:16.360746 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:16.360718 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:16.360799 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:16.360791 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:16.360834 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:16.360822 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:17.366993 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:17.366951 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" event={"ID":"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1","Type":"ContainerStarted","Data":"4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533"} Jul 22 04:11:18.230966 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:18.230923 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:11:18.231277 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:18.231251 2564 patch_prober.go:28] interesting pod/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 04:11:18.231368 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:18.231319 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 04:11:19.231231 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:19.231197 2564 patch_prober.go:28] interesting pod/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 04:11:19.231617 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:19.231251 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 04:11:20.231760 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:20.231725 2564 patch_prober.go:28] interesting pod/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 04:11:20.232238 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:20.231804 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 04:11:21.231258 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.231225 2564 patch_prober.go:28] interesting pod/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 22 04:11:21.231493 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.231286 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 22 04:11:21.398168 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.398122 2564 generic.go:356] "Generic (PLEG): container finished" podID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerID="47b5729ceecfa3d03f50136ae5a37de6d8d5fa3b96e7f31a3720d716338f4349" exitCode=0 Jul 22 04:11:21.398655 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.398208 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" event={"ID":"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084","Type":"ContainerDied","Data":"47b5729ceecfa3d03f50136ae5a37de6d8d5fa3b96e7f31a3720d716338f4349"} Jul 22 04:11:21.398719 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.398700 2564 scope.go:117] "RemoveContainer" containerID="47b5729ceecfa3d03f50136ae5a37de6d8d5fa3b96e7f31a3720d716338f4349" Jul 22 04:11:21.400212 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.400172 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:21.400313 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.400262 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:21.400400 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:21.400310 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:11:22.234426 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.234391 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:11:22.234675 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.234656 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:11:22.235279 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.235261 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:11:22.307659 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.307455 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:22.407886 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.407850 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" event={"ID":"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084","Type":"ContainerStarted","Data":"8496e744bfb408191934e614a0234d98c99bc8370b843ca5120f913b399a3b9b"} Jul 22 04:11:22.783239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.783151 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:22.785288 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.784943 2564 patch_prober.go:28] interesting pod/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 04:11:22.785288 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:22.785017 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 04:11:23.788755 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:23.788720 2564 patch_prober.go:28] interesting pod/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 04:11:23.789319 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:23.789288 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 04:11:24.783568 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:24.783532 2564 patch_prober.go:28] interesting pod/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 04:11:24.783845 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:24.783817 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 04:11:25.783470 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:25.783436 2564 patch_prober.go:28] interesting pod/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body= Jul 22 04:11:25.783948 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:25.783499 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" Jul 22 04:11:26.786250 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:26.786223 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:26.786628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:26.786584 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:26.787144 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:26.787126 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:11:47.597825 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:47.597791 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz"] Jul 22 04:11:47.598321 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:47.597984 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" containerID="cri-o://4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533" gracePeriod=30 Jul 22 04:11:52.236851 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.236813 2564 patch_prober.go:28] interesting pod/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 04:11:52.237305 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.236885 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:11:52.858257 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.858233 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:11:52.913933 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.913865 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-token\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.913933 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.913897 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-credential-socket\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.913933 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.913918 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-socket\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.913933 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.913933 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-certs\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.914239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.913952 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-envoy\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.914239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.913976 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-podinfo\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.914239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914036 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istiod-ca-cert\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.914239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914066 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mgspc\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-kube-api-access-mgspc\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.914239 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914112 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-data\") pod \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\" (UID: \"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1\") " Jul 22 04:11:52.914534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914227 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:52.914534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914239 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:52.914534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914250 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:52.914534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914503 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:52.914534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914526 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:52.914801 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914540 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:52.914801 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.914571 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:11:52.915307 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.915280 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:52.916092 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.916069 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:11:52.916210 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.916191 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:11:52.916317 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.916300 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:11:52.916443 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:52.916425 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-kube-api-access-mgspc" (OuterVolumeSpecName: "kube-api-access-mgspc") pod "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" (UID: "5a7addfe-8cf0-4ddd-a2b4-41388281bcb1"). InnerVolumeSpecName "kube-api-access-mgspc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:11:53.015510 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.015479 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mgspc\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-kube-api-access-mgspc\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:53.015510 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.015507 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:53.015510 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.015516 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:53.015719 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.015524 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:53.015719 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.015532 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:53.015719 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.015539 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:11:53.583896 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.583865 2564 generic.go:356] "Generic (PLEG): container finished" podID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerID="4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533" exitCode=0 Jul 22 04:11:53.584370 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.583910 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" Jul 22 04:11:53.584370 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.583945 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" event={"ID":"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1","Type":"ContainerDied","Data":"4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533"} Jul 22 04:11:53.584370 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.584004 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz" event={"ID":"5a7addfe-8cf0-4ddd-a2b4-41388281bcb1","Type":"ContainerDied","Data":"9dd5ffdf62b141b40cafe180b2c356833fee9894d985c9692cc638947e2d5a46"} Jul 22 04:11:53.584370 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.584027 2564 scope.go:117] "RemoveContainer" containerID="4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533" Jul 22 04:11:53.594255 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.594239 2564 scope.go:117] "RemoveContainer" containerID="5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c" Jul 22 04:11:53.602254 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.602237 2564 scope.go:117] "RemoveContainer" containerID="4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533" Jul 22 04:11:53.602530 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:11:53.602513 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533\": container with ID starting with 4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533 not found: ID does not exist" containerID="4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533" Jul 22 04:11:53.602608 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.602539 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533"} err="failed to get container status \"4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533\": rpc error: code = NotFound desc = could not find container \"4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533\": container with ID starting with 4cc4a476dbdadc73a873641b7c91e0c9b478c9bbfd8a7b442a25a03750ee4533 not found: ID does not exist" Jul 22 04:11:53.602608 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.602562 2564 scope.go:117] "RemoveContainer" containerID="5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c" Jul 22 04:11:53.602782 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:11:53.602765 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c\": container with ID starting with 5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c not found: ID does not exist" containerID="5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c" Jul 22 04:11:53.602828 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.602789 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c"} err="failed to get container status \"5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c\": rpc error: code = NotFound desc = could not find container \"5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c\": container with ID starting with 5573455cbe3c5bbba32eea47bd7732ba16b82ed307c9918a7b9937f1b338545c not found: ID does not exist" Jul 22 04:11:53.607692 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.607674 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz"] Jul 22 04:11:53.614297 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.614274 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-b721b6-openshift-default-645d6bff55-xc6lz"] Jul 22 04:11:53.744762 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:11:53.744728 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" path="/var/lib/kubelet/pods/5a7addfe-8cf0-4ddd-a2b4-41388281bcb1/volumes" Jul 22 04:12:14.503671 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:14.503639 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72"] Jul 22 04:12:14.504253 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:14.503865 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" containerID="cri-o://8496e744bfb408191934e614a0234d98c99bc8370b843ca5120f913b399a3b9b" gracePeriod=30 Jul 22 04:12:15.413580 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.413547 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs"] Jul 22 04:12:15.414020 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.413998 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" Jul 22 04:12:15.414020 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.414019 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" Jul 22 04:12:15.414190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.414042 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" Jul 22 04:12:15.414190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.414051 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="04ac0787-0382-45ca-9604-12dacb0b40dd" containerName="istio-proxy" Jul 22 04:12:15.414190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.414164 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" Jul 22 04:12:15.414190 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.414181 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" Jul 22 04:12:15.416467 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.416450 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.419650 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.419629 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-dd1671-openshift-default-dockercfg-xblrx\"" Jul 22 04:12:15.431700 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.431679 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs"] Jul 22 04:12:15.510583 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510549 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-podinfo\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.510583 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510586 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-socket\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.511095 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510623 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-envoy\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.511095 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510683 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-token\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.511095 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510710 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-certs\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.511095 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510751 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-credential-socket\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.511095 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510782 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z49c\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-kube-api-access-9z49c\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.511095 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510853 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-data\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.511095 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.510898 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istiod-ca-cert\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.611618 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.611583 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z49c\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-kube-api-access-9z49c\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.611827 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.611640 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-data\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.611827 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.611746 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istiod-ca-cert\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.611827 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.611791 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-podinfo\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.611998 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.611926 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-socket\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612054 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612011 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-envoy\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612054 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612033 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-data\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612159 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612070 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-token\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612159 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612101 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-certs\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612159 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612149 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-credential-socket\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612314 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612221 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-socket\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612491 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612472 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-certs\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612569 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612516 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-credential-socket\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.612627 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.612580 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istiod-ca-cert\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.614919 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.614902 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-envoy\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.615224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.615203 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-podinfo\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.622639 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.622582 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-token\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.622639 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.622619 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z49c\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-kube-api-access-9z49c\") pod \"e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.726539 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.726479 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:15.865913 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.865884 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs"] Jul 22 04:12:15.866859 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:12:15.866825 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeccd15df_f5c2_4f2b_a8ea_cc1085b09c9a.slice/crio-059c050d3a0fbbf9f1e2633ccc9f65a49d2e22adcb31688d7a7efa6d630cc4a4 WatchSource:0}: Error finding container 059c050d3a0fbbf9f1e2633ccc9f65a49d2e22adcb31688d7a7efa6d630cc4a4: Status 404 returned error can't find the container with id 059c050d3a0fbbf9f1e2633ccc9f65a49d2e22adcb31688d7a7efa6d630cc4a4 Jul 22 04:12:15.869059 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.869014 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:15.869128 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.869110 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:15.869172 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:15.869156 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:16.709736 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:16.709701 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" event={"ID":"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a","Type":"ContainerStarted","Data":"d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd"} Jul 22 04:12:16.710187 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:16.709759 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" event={"ID":"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a","Type":"ContainerStarted","Data":"059c050d3a0fbbf9f1e2633ccc9f65a49d2e22adcb31688d7a7efa6d630cc4a4"} Jul 22 04:12:16.726734 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:16.726702 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:16.731467 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:16.731437 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:16.733051 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:16.733011 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" podStartSLOduration=1.732998365 podStartE2EDuration="1.732998365s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:12:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:12:16.731736791 +0000 UTC m=+2323.525225284" watchObservedRunningTime="2026-07-22 04:12:16.732998365 +0000 UTC m=+2323.526486855" Jul 22 04:12:17.714981 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:17.714945 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:17.715707 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:17.715690 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:19.728052 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.728019 2564 generic.go:356] "Generic (PLEG): container finished" podID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerID="8496e744bfb408191934e614a0234d98c99bc8370b843ca5120f913b399a3b9b" exitCode=0 Jul 22 04:12:19.728548 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.728091 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" event={"ID":"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084","Type":"ContainerDied","Data":"8496e744bfb408191934e614a0234d98c99bc8370b843ca5120f913b399a3b9b"} Jul 22 04:12:19.728548 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.728171 2564 scope.go:117] "RemoveContainer" containerID="47b5729ceecfa3d03f50136ae5a37de6d8d5fa3b96e7f31a3720d716338f4349" Jul 22 04:12:19.763511 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.763440 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:12:19.847664 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.847627 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-token\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.847836 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.847674 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-envoy\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.847836 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.847703 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-podinfo\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.847836 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.847718 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-credential-socket\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.847947 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.847902 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fkjkv\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-kube-api-access-fkjkv\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.848003 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.847951 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-socket\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.848003 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.847992 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istiod-ca-cert\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.848110 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848024 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-data\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.848110 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848049 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-certs\") pod \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\" (UID: \"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084\") " Jul 22 04:12:19.848110 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848052 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:12:19.848261 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848220 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:12:19.849030 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848721 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.849030 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848749 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.849030 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848755 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:12:19.849030 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.848890 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-data" (OuterVolumeSpecName: "istio-data") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:12:19.849357 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.849072 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:12:19.850261 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.850235 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:12:19.850380 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.850295 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-kube-api-access-fkjkv" (OuterVolumeSpecName: "kube-api-access-fkjkv") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "kube-api-access-fkjkv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:12:19.850496 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.850472 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:12:19.850812 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.850791 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-token" (OuterVolumeSpecName: "istio-token") pod "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" (UID: "56676aba-bcd7-4dfa-98c7-bb1f5f7e1084"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:12:19.949785 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.949741 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.949785 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.949780 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fkjkv\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-kube-api-access-fkjkv\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.949992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.949792 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.949992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.949804 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.949992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.949814 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.949992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.949824 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:19.949992 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:19.949833 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:12:20.734664 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:20.734634 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" Jul 22 04:12:20.734664 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:20.734656 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72" event={"ID":"56676aba-bcd7-4dfa-98c7-bb1f5f7e1084","Type":"ContainerDied","Data":"49315a21e66b2fba5ab9818f7723894a4c4ca7f6078c8080522d5a6d3660e0b4"} Jul 22 04:12:20.735159 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:20.734710 2564 scope.go:117] "RemoveContainer" containerID="8496e744bfb408191934e614a0234d98c99bc8370b843ca5120f913b399a3b9b" Jul 22 04:12:20.763301 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:20.763271 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72"] Jul 22 04:12:20.769265 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:20.769238 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-656735-openshift-default-57bf8d9cd9-9pz72"] Jul 22 04:12:21.744831 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:21.744798 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" path="/var/lib/kubelet/pods/56676aba-bcd7-4dfa-98c7-bb1f5f7e1084/volumes" Jul 22 04:12:23.812961 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.812927 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6"] Jul 22 04:12:23.813402 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813287 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" Jul 22 04:12:23.813402 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813299 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" Jul 22 04:12:23.813402 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813318 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" Jul 22 04:12:23.813402 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813324 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" Jul 22 04:12:23.813402 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813373 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" Jul 22 04:12:23.813402 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813379 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a7addfe-8cf0-4ddd-a2b4-41388281bcb1" containerName="istio-proxy" Jul 22 04:12:23.813656 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813450 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" Jul 22 04:12:23.813656 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.813457 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="56676aba-bcd7-4dfa-98c7-bb1f5f7e1084" containerName="istio-proxy" Jul 22 04:12:23.816673 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.816655 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.821109 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.821090 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-38610b-openshift-default-dockercfg-6kwqj\"" Jul 22 04:12:23.830200 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.830178 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6"] Jul 22 04:12:23.885046 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885010 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-envoy\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885207 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885061 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-data\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885207 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885082 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istiod-ca-cert\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885207 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885135 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-token\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885207 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885161 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-podinfo\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885401 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885235 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msbfw\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-kube-api-access-msbfw\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885401 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885271 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-certs\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885401 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885293 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-socket\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.885401 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.885307 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-credential-socket\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.986750 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.986716 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-envoy\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.986962 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.986783 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-data\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.986962 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.986821 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istiod-ca-cert\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.986962 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.986853 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-token\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.986962 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.986886 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-podinfo\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.986962 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.986944 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msbfw\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-kube-api-access-msbfw\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.987658 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.986976 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-certs\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.987658 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.987007 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-socket\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.987658 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.987030 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-credential-socket\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.987658 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.987299 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-data\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.987658 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.987510 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-certs\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.987658 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.987553 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-socket\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.987658 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.987610 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-credential-socket\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.988103 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.988080 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istiod-ca-cert\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.989370 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.989350 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-envoy\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.989616 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.989602 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-podinfo\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.995585 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.995560 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-token\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:23.998602 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:23.998582 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msbfw\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-kube-api-access-msbfw\") pod \"e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:24.126498 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.126378 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:24.261719 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.261690 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6"] Jul 22 04:12:24.262999 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:12:24.262976 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4f91dd20_bd79_47f6_99ce_7b3adb7d479d.slice/crio-4d32a9daf41d646f5f03d6778314ab4e55bfe6b7aeb8730c6076bca6d38cbcca WatchSource:0}: Error finding container 4d32a9daf41d646f5f03d6778314ab4e55bfe6b7aeb8730c6076bca6d38cbcca: Status 404 returned error can't find the container with id 4d32a9daf41d646f5f03d6778314ab4e55bfe6b7aeb8730c6076bca6d38cbcca Jul 22 04:12:24.264951 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.264917 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:24.265046 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.264991 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:24.265046 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.265036 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:24.756706 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.756672 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" event={"ID":"4f91dd20-bd79-47f6-99ce-7b3adb7d479d","Type":"ContainerStarted","Data":"05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856"} Jul 22 04:12:24.756871 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.756724 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" event={"ID":"4f91dd20-bd79-47f6-99ce-7b3adb7d479d","Type":"ContainerStarted","Data":"4d32a9daf41d646f5f03d6778314ab4e55bfe6b7aeb8730c6076bca6d38cbcca"} Jul 22 04:12:24.780428 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:24.780383 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podStartSLOduration=1.780368395 podStartE2EDuration="1.780368395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 04:12:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:12:24.778608643 +0000 UTC m=+2331.572097159" watchObservedRunningTime="2026-07-22 04:12:24.780368395 +0000 UTC m=+2331.573856887" Jul 22 04:12:25.127391 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:25.127287 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:25.131766 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:25.131742 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:25.761780 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:25.761741 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:25.762573 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:25.762555 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:33.756496 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:33.756451 2564 patch_prober.go:28] interesting pod/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 22 04:12:33.756880 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:33.756699 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 22 04:12:33.958777 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:33.958745 2564 scope.go:117] "RemoveContainer" containerID="0a48ee028172e208639a2423bfa561a4b59c4f64b339ed3ccc33faa498c05ddb" Jul 22 04:12:34.818031 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:34.817939 2564 generic.go:356] "Generic (PLEG): container finished" podID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerID="d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd" exitCode=0 Jul 22 04:12:34.818031 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:34.818015 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" event={"ID":"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a","Type":"ContainerDied","Data":"d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd"} Jul 22 04:12:34.818609 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:34.818487 2564 scope.go:117] "RemoveContainer" containerID="d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd" Jul 22 04:12:34.819532 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:34.819494 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:34.819633 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:34.819582 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:34.819633 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:34.819626 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:35.835628 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:35.835591 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" event={"ID":"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a","Type":"ContainerStarted","Data":"ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b"} Jul 22 04:12:36.727264 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:36.727238 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:36.727589 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:36.727460 2564 patch_prober.go:28] interesting pod/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 22 04:12:36.727721 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:36.727599 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 22 04:12:37.727663 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:37.727626 2564 patch_prober.go:28] interesting pod/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 22 04:12:37.728222 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:37.728194 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 22 04:12:38.727132 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:38.727087 2564 patch_prober.go:28] interesting pod/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 22 04:12:38.727361 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:38.727153 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 22 04:12:39.730695 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:39.730668 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:39.731108 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:39.731062 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:39.731868 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:39.731849 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:12:40.763774 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.763744 2564 patch_prober.go:28] interesting pod/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 22 04:12:40.764209 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.763809 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 22 04:12:40.864134 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.864109 2564 generic.go:356] "Generic (PLEG): container finished" podID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerID="05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856" exitCode=0 Jul 22 04:12:40.864283 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.864175 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" event={"ID":"4f91dd20-bd79-47f6-99ce-7b3adb7d479d","Type":"ContainerDied","Data":"05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856"} Jul 22 04:12:40.864675 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.864662 2564 scope.go:117] "RemoveContainer" containerID="05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856" Jul 22 04:12:40.865774 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.865735 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:40.865869 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.865851 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:40.865919 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:40.865903 2564 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 22 04:12:41.870648 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:41.870606 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" event={"ID":"4f91dd20-bd79-47f6-99ce-7b3adb7d479d","Type":"ContainerStarted","Data":"4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f"} Jul 22 04:12:42.127024 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:42.126924 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:42.127176 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:42.127037 2564 patch_prober.go:28] interesting pod/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 22 04:12:42.127176 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:42.127102 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 22 04:12:43.126891 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:43.126855 2564 patch_prober.go:28] interesting pod/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 22 04:12:43.127293 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:43.126905 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 22 04:12:44.127031 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:44.126995 2564 patch_prober.go:28] interesting pod/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 22 04:12:44.127423 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:44.127045 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 22 04:12:45.127674 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:45.127639 2564 patch_prober.go:28] interesting pod/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 22 04:12:45.128064 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:45.127691 2564 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 22 04:12:46.131245 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:46.131217 2564 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:46.131828 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:46.131534 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:12:46.131965 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:12:46.131946 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:15:00.162639 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:00.162603 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v"] Jul 22 04:15:00.165020 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:00.165005 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v" Jul 22 04:15:00.168692 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:00.168669 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rbfm6\"" Jul 22 04:15:00.174290 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:00.174270 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v" Jul 22 04:15:00.188936 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:00.188913 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v"] Jul 22 04:15:00.516004 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:00.515979 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v"] Jul 22 04:15:00.517502 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:15:00.517460 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43dc1a5d_73c9_4513_b147_76a3052a2cd4.slice/crio-3c002cac8821721042ed54b05f36824a2d42ba2b3305b3d8c89c5e13019066e1 WatchSource:0}: Error finding container 3c002cac8821721042ed54b05f36824a2d42ba2b3305b3d8c89c5e13019066e1: Status 404 returned error can't find the container with id 3c002cac8821721042ed54b05f36824a2d42ba2b3305b3d8c89c5e13019066e1 Jul 22 04:15:00.615417 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:00.615384 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v" event={"ID":"43dc1a5d-73c9-4513-b147-76a3052a2cd4","Type":"ContainerStarted","Data":"3c002cac8821721042ed54b05f36824a2d42ba2b3305b3d8c89c5e13019066e1"} Jul 22 04:15:01.622327 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:01.622292 2564 generic.go:356] "Generic (PLEG): container finished" podID="43dc1a5d-73c9-4513-b147-76a3052a2cd4" containerID="c747e274ec9a3904cc598f599f44ea083702d22e4bc3f80900e39b1af8a56666" exitCode=0 Jul 22 04:15:01.622695 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:01.622384 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v" event={"ID":"43dc1a5d-73c9-4513-b147-76a3052a2cd4","Type":"ContainerDied","Data":"c747e274ec9a3904cc598f599f44ea083702d22e4bc3f80900e39b1af8a56666"} Jul 22 04:15:02.761396 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:02.761369 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v" Jul 22 04:15:03.635456 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:03.635430 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v" Jul 22 04:15:03.635629 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:03.635470 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v" event={"ID":"43dc1a5d-73c9-4513-b147-76a3052a2cd4","Type":"ContainerDied","Data":"3c002cac8821721042ed54b05f36824a2d42ba2b3305b3d8c89c5e13019066e1"} Jul 22 04:15:03.635629 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:15:03.635517 2564 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3c002cac8821721042ed54b05f36824a2d42ba2b3305b3d8c89c5e13019066e1" Jul 22 04:16:34.014021 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:16:34.013987 2564 scope.go:117] "RemoveContainer" containerID="fbf5c0ad9ddf0cc782d8e14b6293d1f244f2833dd313a0cf1a386c96cc97488c" Jul 22 04:18:41.373062 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:41.372981 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v"] Jul 22 04:18:41.376574 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:41.376547 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-dd1671-29744895-jbq6v"] Jul 22 04:18:41.745688 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:41.745653 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43dc1a5d-73c9-4513-b147-76a3052a2cd4" path="/var/lib/kubelet/pods/43dc1a5d-73c9-4513-b147-76a3052a2cd4/volumes" Jul 22 04:18:53.226733 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:53.226694 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs"] Jul 22 04:18:53.227156 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:53.226923 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" containerID="cri-o://ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b" gracePeriod=30 Jul 22 04:18:54.732994 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:54.732960 2564 patch_prober.go:28] interesting pod/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 04:18:54.733408 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:54.733013 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:18:58.482363 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.482316 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:18:58.602442 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602398 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-credential-socket\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602442 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602447 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-socket\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602682 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602480 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istiod-ca-cert\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602682 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602499 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-token\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602682 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602528 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-data\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602682 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602550 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-certs\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602878 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602686 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:18:58.602878 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602716 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-podinfo\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602878 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602788 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-envoy\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602878 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602843 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z49c\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-kube-api-access-9z49c\") pod \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\" (UID: \"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a\") " Jul 22 04:18:58.602878 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602855 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:18:58.602878 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602870 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:18:58.603163 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.602951 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:18:58.603224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.603166 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.603224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.603185 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.603224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.603197 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.603224 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.603206 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.603802 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.603773 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-data" (OuterVolumeSpecName: "istio-data") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:18:58.605009 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.604984 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:18:58.605188 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.605065 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-token" (OuterVolumeSpecName: "istio-token") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:18:58.605242 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.605190 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-kube-api-access-9z49c" (OuterVolumeSpecName: "kube-api-access-9z49c") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "kube-api-access-9z49c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:18:58.605242 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.605193 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" (UID: "eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:18:58.703787 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.703750 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.703787 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.703782 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.703787 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.703792 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.703953 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.703800 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z49c\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-kube-api-access-9z49c\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.703953 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.703809 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:18:58.888842 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.888777 2564 generic.go:356] "Generic (PLEG): container finished" podID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerID="ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b" exitCode=0 Jul 22 04:18:58.888842 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.888833 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" Jul 22 04:18:58.888842 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.888836 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" event={"ID":"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a","Type":"ContainerDied","Data":"ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b"} Jul 22 04:18:58.889064 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.888891 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs" event={"ID":"eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a","Type":"ContainerDied","Data":"059c050d3a0fbbf9f1e2633ccc9f65a49d2e22adcb31688d7a7efa6d630cc4a4"} Jul 22 04:18:58.889064 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.888924 2564 scope.go:117] "RemoveContainer" containerID="ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b" Jul 22 04:18:58.898700 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.898683 2564 scope.go:117] "RemoveContainer" containerID="d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd" Jul 22 04:18:58.907622 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.907606 2564 scope.go:117] "RemoveContainer" containerID="ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b" Jul 22 04:18:58.907901 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:18:58.907884 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b\": container with ID starting with ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b not found: ID does not exist" containerID="ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b" Jul 22 04:18:58.907956 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.907909 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b"} err="failed to get container status \"ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b\": rpc error: code = NotFound desc = could not find container \"ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b\": container with ID starting with ae57759f1c56d783f6b36d523714911f4c1230bcd115a378a15786b224a6de1b not found: ID does not exist" Jul 22 04:18:58.907956 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.907926 2564 scope.go:117] "RemoveContainer" containerID="d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd" Jul 22 04:18:58.908151 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:18:58.908133 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd\": container with ID starting with d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd not found: ID does not exist" containerID="d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd" Jul 22 04:18:58.908194 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.908157 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd"} err="failed to get container status \"d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd\": rpc error: code = NotFound desc = could not find container \"d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd\": container with ID starting with d6a7642d927186d521018716e793634334047eebfecc09f5f080baa107d7a4dd not found: ID does not exist" Jul 22 04:18:58.911902 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.911882 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs"] Jul 22 04:18:58.916053 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:58.916025 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-dd1671-openshift-default-687564dd95-qw5bs"] Jul 22 04:18:59.746743 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:18:59.746707 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" path="/var/lib/kubelet/pods/eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a/volumes" Jul 22 04:19:15.358214 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:15.358177 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6"] Jul 22 04:19:15.358767 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:15.358405 2564 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" containerID="cri-o://4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f" gracePeriod=30 Jul 22 04:19:16.133871 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:16.133834 2564 patch_prober.go:28] interesting pod/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 22 04:19:16.134044 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:16.133893 2564 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 22 04:19:20.618403 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.618305 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:19:20.694823 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.694791 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-token\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.694979 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.694833 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-certs\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.694979 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.694859 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-credential-socket\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.694979 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.694892 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-socket\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.694979 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.694920 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msbfw\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-kube-api-access-msbfw\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.694979 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.694945 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-podinfo\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.695235 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695040 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-data\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.695235 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695072 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istiod-ca-cert\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.695235 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695110 2564 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-envoy\") pod \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\" (UID: \"4f91dd20-bd79-47f6-99ce-7b3adb7d479d\") " Jul 22 04:19:20.695235 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695105 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:19:20.695235 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695174 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:19:20.695235 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695184 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:19:20.695557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695417 2564 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-certs\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.695557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695427 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 04:19:20.695557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695436 2564 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-credential-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.695557 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.695452 2564 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-workload-socket\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.696138 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.696110 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-data" (OuterVolumeSpecName: "istio-data") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:19:20.697260 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.697237 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-kube-api-access-msbfw" (OuterVolumeSpecName: "kube-api-access-msbfw") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "kube-api-access-msbfw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:19:20.697260 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.697244 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-token" (OuterVolumeSpecName: "istio-token") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 04:19:20.697676 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.697648 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 22 04:19:20.697787 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.697728 2564 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4f91dd20-bd79-47f6-99ce-7b3adb7d479d" (UID: "4f91dd20-bd79-47f6-99ce-7b3adb7d479d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 04:19:20.796344 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.796312 2564 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-token\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.796534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.796368 2564 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msbfw\" (UniqueName: \"kubernetes.io/projected/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-kube-api-access-msbfw\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.796534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.796383 2564 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-podinfo\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.796534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.796396 2564 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-data\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.796534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.796409 2564 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istiod-ca-cert\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:20.796534 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:20.796420 2564 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4f91dd20-bd79-47f6-99ce-7b3adb7d479d-istio-envoy\") on node \"ip-10-0-139-52.ec2.internal\" DevicePath \"\"" Jul 22 04:19:21.011787 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.011754 2564 generic.go:356] "Generic (PLEG): container finished" podID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerID="4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f" exitCode=0 Jul 22 04:19:21.011970 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.011803 2564 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" Jul 22 04:19:21.011970 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.011847 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" event={"ID":"4f91dd20-bd79-47f6-99ce-7b3adb7d479d","Type":"ContainerDied","Data":"4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f"} Jul 22 04:19:21.011970 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.011905 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6" event={"ID":"4f91dd20-bd79-47f6-99ce-7b3adb7d479d","Type":"ContainerDied","Data":"4d32a9daf41d646f5f03d6778314ab4e55bfe6b7aeb8730c6076bca6d38cbcca"} Jul 22 04:19:21.011970 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.011930 2564 scope.go:117] "RemoveContainer" containerID="4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f" Jul 22 04:19:21.021686 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.021664 2564 scope.go:117] "RemoveContainer" containerID="05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856" Jul 22 04:19:21.030254 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.030234 2564 scope.go:117] "RemoveContainer" containerID="4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f" Jul 22 04:19:21.030547 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:19:21.030527 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f\": container with ID starting with 4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f not found: ID does not exist" containerID="4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f" Jul 22 04:19:21.030627 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.030554 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f"} err="failed to get container status \"4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f\": rpc error: code = NotFound desc = could not find container \"4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f\": container with ID starting with 4c0c0f4d92f2c976c22985eddd405d3cf20ac09b17b79b6eb2847aa20520e53f not found: ID does not exist" Jul 22 04:19:21.030627 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.030571 2564 scope.go:117] "RemoveContainer" containerID="05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856" Jul 22 04:19:21.030819 ip-10-0-139-52 kubenswrapper[2564]: E0722 04:19:21.030802 2564 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856\": container with ID starting with 05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856 not found: ID does not exist" containerID="05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856" Jul 22 04:19:21.030878 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.030828 2564 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856"} err="failed to get container status \"05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856\": rpc error: code = NotFound desc = could not find container \"05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856\": container with ID starting with 05e0b8b8fe47c1a40b1f129b68916645e4801fe8fc95045b5d434fc6371bb856 not found: ID does not exist" Jul 22 04:19:21.047008 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.046985 2564 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6"] Jul 22 04:19:21.050024 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.050002 2564 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-38610b-openshift-default-6bf86cbbbc-hlfg6"] Jul 22 04:19:21.745699 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.745666 2564 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" path="/var/lib/kubelet/pods/4f91dd20-bd79-47f6-99ce-7b3adb7d479d/volumes" Jul 22 04:19:21.865763 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:21.865737 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74d8745f54-ng7b4_530f4bcb-7050-41f0-ac8d-47f44b676a7b/manager/0.log" Jul 22 04:19:22.220568 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:22.220480 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-b4xqx_79b4fd24-783d-4058-b67c-90fb163a28a2/copy-manifests/0.log" Jul 22 04:19:22.227424 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:22.227399 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-b4xqx_79b4fd24-783d-4058-b67c-90fb163a28a2/manager/0.log" Jul 22 04:19:22.951449 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:22.951419 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744865-twrw2_db7b71fe-da49-4a3a-bab5-be83a122a20a/cleanup/0.log" Jul 22 04:19:23.061455 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:23.061427 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744880-n5bdq_a1230403-f01c-4743-96e8-54364c98fd76/cleanup/0.log" Jul 22 04:19:24.070705 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.070673 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r_9c456b6b-2ce6-45c8-8250-aa8d48efb99b/util/0.log" Jul 22 04:19:24.077524 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.077500 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r_9c456b6b-2ce6-45c8-8250-aa8d48efb99b/pull/0.log" Jul 22 04:19:24.083399 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.083374 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r_9c456b6b-2ce6-45c8-8250-aa8d48efb99b/extract/0.log" Jul 22 04:19:24.197308 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.197279 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5_ffc675d9-96a6-487c-b309-b28cd00e46f6/util/0.log" Jul 22 04:19:24.203346 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.203300 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5_ffc675d9-96a6-487c-b309-b28cd00e46f6/pull/0.log" Jul 22 04:19:24.209903 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.209881 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5_ffc675d9-96a6-487c-b309-b28cd00e46f6/extract/0.log" Jul 22 04:19:24.326223 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.326143 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z_75de5bed-620f-49c6-8053-5c805123dd06/util/0.log" Jul 22 04:19:24.331874 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.331854 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z_75de5bed-620f-49c6-8053-5c805123dd06/pull/0.log" Jul 22 04:19:24.337686 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.337668 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z_75de5bed-620f-49c6-8053-5c805123dd06/extract/0.log" Jul 22 04:19:24.447107 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.447081 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4_ac457ee9-3401-40e5-b31c-ed84f789875b/util/0.log" Jul 22 04:19:24.452932 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.452911 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4_ac457ee9-3401-40e5-b31c-ed84f789875b/pull/0.log" Jul 22 04:19:24.458275 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:24.458259 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4_ac457ee9-3401-40e5-b31c-ed84f789875b/extract/0.log" Jul 22 04:19:25.777195 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:25.777164 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47mgtz_3a5870d1-e07b-48ed-bd97-ba6406dc054c/istio-proxy/0.log" Jul 22 04:19:25.901532 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:25.901502 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ck86j_d0af4091-4932-43c9-924e-9f17470da231/discovery/0.log" Jul 22 04:19:26.246214 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:26.246180 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vrktd_5ab6787b-526f-438f-935b-8a400e7c4b68/istio-proxy/1.log" Jul 22 04:19:26.478894 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:26.478868 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l4skc_95b1b413-599a-474c-8987-a7786d3c6817/payload-processing/0.log" Jul 22 04:19:26.942191 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:26.942163 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qjnf4_b0cb81ef-1f92-48c2-9fda-a532a33e70ad/main/0.log" Jul 22 04:19:35.877015 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:35.876977 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-k9vqm_200a8fe2-eab4-4e9f-924f-03db74707208/global-pull-secret-syncer/0.log" Jul 22 04:19:36.047971 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:36.047933 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xxkvv_8590ac76-bbde-4b05-84c1-f3b01dfe0a13/konnectivity-agent/0.log" Jul 22 04:19:36.104027 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:36.103996 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-52.ec2.internal_bd39ce5fb20c3f1559561494fcf3f4a1/haproxy/0.log" Jul 22 04:19:40.318257 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.318223 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r_9c456b6b-2ce6-45c8-8250-aa8d48efb99b/extract/0.log" Jul 22 04:19:40.339998 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.339978 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r_9c456b6b-2ce6-45c8-8250-aa8d48efb99b/util/0.log" Jul 22 04:19:40.359415 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.359396 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759k759r_9c456b6b-2ce6-45c8-8250-aa8d48efb99b/pull/0.log" Jul 22 04:19:40.384141 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.384119 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5_ffc675d9-96a6-487c-b309-b28cd00e46f6/extract/0.log" Jul 22 04:19:40.409929 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.409902 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5_ffc675d9-96a6-487c-b309-b28cd00e46f6/util/0.log" Jul 22 04:19:40.434995 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.434970 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0txrk5_ffc675d9-96a6-487c-b309-b28cd00e46f6/pull/0.log" Jul 22 04:19:40.473250 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.473231 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z_75de5bed-620f-49c6-8053-5c805123dd06/extract/0.log" Jul 22 04:19:40.507175 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.507155 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z_75de5bed-620f-49c6-8053-5c805123dd06/util/0.log" Jul 22 04:19:40.536972 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.536954 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73v555z_75de5bed-620f-49c6-8053-5c805123dd06/pull/0.log" Jul 22 04:19:40.573256 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.573199 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4_ac457ee9-3401-40e5-b31c-ed84f789875b/extract/0.log" Jul 22 04:19:40.598792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.598775 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4_ac457ee9-3401-40e5-b31c-ed84f789875b/util/0.log" Jul 22 04:19:40.618184 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:40.618168 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1fdgq4_ac457ee9-3401-40e5-b31c-ed84f789875b/pull/0.log" Jul 22 04:19:43.162547 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.162518 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c839d350-f629-4775-9b9f-b43945f3eddd/alertmanager/0.log" Jul 22 04:19:43.188989 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.188965 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c839d350-f629-4775-9b9f-b43945f3eddd/config-reloader/0.log" Jul 22 04:19:43.217324 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.217298 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c839d350-f629-4775-9b9f-b43945f3eddd/kube-rbac-proxy-web/0.log" Jul 22 04:19:43.241854 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.241816 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c839d350-f629-4775-9b9f-b43945f3eddd/kube-rbac-proxy/0.log" Jul 22 04:19:43.266913 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.266882 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c839d350-f629-4775-9b9f-b43945f3eddd/kube-rbac-proxy-metric/0.log" Jul 22 04:19:43.292961 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.292932 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c839d350-f629-4775-9b9f-b43945f3eddd/prom-label-proxy/0.log" Jul 22 04:19:43.357506 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.357427 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_c839d350-f629-4775-9b9f-b43945f3eddd/init-config-reloader/0.log" Jul 22 04:19:43.534854 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.534830 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-lxhlv_a01b5c89-22e1-47f2-a625-de9c3b7f8681/monitoring-plugin/0.log" Jul 22 04:19:43.651824 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.651765 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzsxx_c3e77fc6-be39-4cd3-8f43-515b89545689/node-exporter/0.log" Jul 22 04:19:43.672306 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.672279 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzsxx_c3e77fc6-be39-4cd3-8f43-515b89545689/kube-rbac-proxy/0.log" Jul 22 04:19:43.699478 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.699461 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzsxx_c3e77fc6-be39-4cd3-8f43-515b89545689/init-textfile/0.log" Jul 22 04:19:43.803823 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.803801 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-rqs7j_784e29f1-92ee-473d-9b46-38564caffd18/kube-rbac-proxy-main/0.log" Jul 22 04:19:43.825216 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.825194 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-rqs7j_784e29f1-92ee-473d-9b46-38564caffd18/kube-rbac-proxy-self/0.log" Jul 22 04:19:43.846630 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:43.846610 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-rqs7j_784e29f1-92ee-473d-9b46-38564caffd18/openshift-state-metrics/0.log" Jul 22 04:19:44.104572 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:44.104548 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-85574b5f49-h2p8d_4ab5aa8d-20e1-48b5-8fa3-53b2f3ebecf8/prometheus-operator-admission-webhook/0.log" Jul 22 04:19:44.130146 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:44.130119 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76586b478c-9bvln_8cb7f48a-f9bd-4ce9-b146-0e7058390e92/telemeter-client/0.log" Jul 22 04:19:44.150161 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:44.150138 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76586b478c-9bvln_8cb7f48a-f9bd-4ce9-b146-0e7058390e92/reload/0.log" Jul 22 04:19:44.169536 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:44.169512 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-76586b478c-9bvln_8cb7f48a-f9bd-4ce9-b146-0e7058390e92/kube-rbac-proxy/0.log" Jul 22 04:19:45.656199 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:45.656167 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-7bc7489cb5-5phsl_e5411e49-cf4d-4c68-96c0-b0d301c58695/networking-console-plugin/0.log" Jul 22 04:19:46.493561 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493528 2564 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng"] Jul 22 04:19:46.493893 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493880 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" Jul 22 04:19:46.493940 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493895 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" Jul 22 04:19:46.493940 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493911 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" Jul 22 04:19:46.493940 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493916 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" Jul 22 04:19:46.493940 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493923 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43dc1a5d-73c9-4513-b147-76a3052a2cd4" containerName="cleanup" Jul 22 04:19:46.493940 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493928 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="43dc1a5d-73c9-4513-b147-76a3052a2cd4" containerName="cleanup" Jul 22 04:19:46.493940 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493937 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" Jul 22 04:19:46.493940 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493942 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" Jul 22 04:19:46.494149 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493949 2564 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" Jul 22 04:19:46.494149 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.493956 2564 state_mem.go:107] "Deleted CPUSet assignment" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" Jul 22 04:19:46.494149 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.494021 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="43dc1a5d-73c9-4513-b147-76a3052a2cd4" containerName="cleanup" Jul 22 04:19:46.494149 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.494028 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="eccd15df-f5c2-4f2b-a8ea-cc1085b09c9a" containerName="istio-proxy" Jul 22 04:19:46.494149 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.494034 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" Jul 22 04:19:46.494149 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.494041 2564 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f91dd20-bd79-47f6-99ce-7b3adb7d479d" containerName="istio-proxy" Jul 22 04:19:46.496285 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.496265 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.499717 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.499685 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m4q6n\"/\"openshift-service-ca.crt\"" Jul 22 04:19:46.499717 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.499696 2564 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-m4q6n\"/\"kube-root-ca.crt\"" Jul 22 04:19:46.499907 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.499697 2564 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-m4q6n\"/\"default-dockercfg-hhvf6\"" Jul 22 04:19:46.506715 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.506693 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng"] Jul 22 04:19:46.609923 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.609897 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-proc\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.610084 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.609949 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-podres\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.610084 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.610012 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-lib-modules\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.610084 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.610032 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-sys\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.610084 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.610062 2564 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbp8q\" (UniqueName: \"kubernetes.io/projected/b26d6480-adc4-41b7-be59-b7f06d67251a-kube-api-access-hbp8q\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711409 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711384 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-proc\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711426 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-podres\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711450 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-lib-modules\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711487 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-sys\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711540 2564 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbp8q\" (UniqueName: \"kubernetes.io/projected/b26d6480-adc4-41b7-be59-b7f06d67251a-kube-api-access-hbp8q\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711557 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-lib-modules\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711566 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-podres\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711573 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-proc\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.711792 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.711608 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b26d6480-adc4-41b7-be59-b7f06d67251a-sys\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.721725 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.721697 2564 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbp8q\" (UniqueName: \"kubernetes.io/projected/b26d6480-adc4-41b7-be59-b7f06d67251a-kube-api-access-hbp8q\") pod \"perf-node-gather-daemonset-vncng\" (UID: \"b26d6480-adc4-41b7-be59-b7f06d67251a\") " pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.807305 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.807231 2564 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:46.851952 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.851926 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-688468586-b8t7m_0bda44b0-94ae-4dd3-8e2a-e71517e164c1/console/0.log" Jul 22 04:19:46.939260 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:46.939236 2564 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng"] Jul 22 04:19:46.941127 ip-10-0-139-52 kubenswrapper[2564]: W0722 04:19:46.941100 2564 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podb26d6480_adc4_41b7_be59_b7f06d67251a.slice/crio-61861f7fe9f8c62965fbc17a9f9f455cfc8245bd32859da342c10aba3228dfbb WatchSource:0}: Error finding container 61861f7fe9f8c62965fbc17a9f9f455cfc8245bd32859da342c10aba3228dfbb: Status 404 returned error can't find the container with id 61861f7fe9f8c62965fbc17a9f9f455cfc8245bd32859da342c10aba3228dfbb Jul 22 04:19:47.157540 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:47.157453 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" event={"ID":"b26d6480-adc4-41b7-be59-b7f06d67251a","Type":"ContainerStarted","Data":"d2882ea18241c0b917d2e8186e0aac7a257e116a16d01a0e13d1833e991e9299"} Jul 22 04:19:47.157540 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:47.157502 2564 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" event={"ID":"b26d6480-adc4-41b7-be59-b7f06d67251a","Type":"ContainerStarted","Data":"61861f7fe9f8c62965fbc17a9f9f455cfc8245bd32859da342c10aba3228dfbb"} Jul 22 04:19:47.157540 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:47.157536 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:47.180151 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:47.180098 2564 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" podStartSLOduration=1.180082206 podStartE2EDuration="1.180082206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 04:19:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 04:19:47.179222241 +0000 UTC m=+2773.972710733" watchObservedRunningTime="2026-07-22 04:19:47.180082206 +0000 UTC m=+2773.973570697" Jul 22 04:19:48.668192 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:48.668157 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xx9rw_670451d7-67ac-46f2-a8ea-af8b553b7602/dns/0.log" Jul 22 04:19:48.687106 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:48.687081 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-xx9rw_670451d7-67ac-46f2-a8ea-af8b553b7602/kube-rbac-proxy/0.log" Jul 22 04:19:48.707160 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:48.707140 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-fqjtc_7fca8f59-2e5c-403a-adba-6d628d831737/dns-node-resolver/0.log" Jul 22 04:19:49.310813 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:49.310781 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4pgwk_db43e671-0bc5-403f-be13-99333644b038/node-ca/0.log" Jul 22 04:19:50.289667 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:50.289640 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47mgtz_3a5870d1-e07b-48ed-bd97-ba6406dc054c/istio-proxy/0.log" Jul 22 04:19:50.413560 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:50.413519 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ck86j_d0af4091-4932-43c9-924e-9f17470da231/discovery/0.log" Jul 22 04:19:50.490610 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:50.490567 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vrktd_5ab6787b-526f-438f-935b-8a400e7c4b68/istio-proxy/0.log" Jul 22 04:19:50.519837 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:50.519803 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vrktd_5ab6787b-526f-438f-935b-8a400e7c4b68/istio-proxy/1.log" Jul 22 04:19:50.585496 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:50.585405 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l4skc_95b1b413-599a-474c-8987-a7786d3c6817/payload-processing/0.log" Jul 22 04:19:51.387433 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:51.387383 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tgn8x_66905f1b-1a5f-49f7-9411-2123829bdb6c/serve-healthcheck-canary/0.log" Jul 22 04:19:51.862851 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:51.862814 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dxbf7_79027f34-2de1-4e27-9422-43081303a8d8/kube-rbac-proxy/0.log" Jul 22 04:19:51.884304 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:51.884264 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dxbf7_79027f34-2de1-4e27-9422-43081303a8d8/exporter/0.log" Jul 22 04:19:51.904571 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:51.904533 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-dxbf7_79027f34-2de1-4e27-9422-43081303a8d8/extractor/0.log" Jul 22 04:19:53.178885 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:53.178854 2564 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-m4q6n/perf-node-gather-daemonset-vncng" Jul 22 04:19:54.444746 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:54.444718 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-74d8745f54-ng7b4_530f4bcb-7050-41f0-ac8d-47f44b676a7b/manager/0.log" Jul 22 04:19:54.552246 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:54.552221 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-b4xqx_79b4fd24-783d-4058-b67c-90fb163a28a2/manager/0.log" Jul 22 04:19:54.573138 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:54.573119 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5fd47b996b-b4xqx_79b4fd24-783d-4058-b67c-90fb163a28a2/copy-manifests/0.log" Jul 22 04:19:56.118309 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:56.118278 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78585bcb5c-fmc8r_ec6419df-c83c-44ff-83b8-87a8e4bae5e4/manager/0.log" Jul 22 04:19:56.745545 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:56.745501 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47mgtz_3a5870d1-e07b-48ed-bd97-ba6406dc054c/istio-proxy/0.log" Jul 22 04:19:56.887597 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:56.887563 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ck86j_d0af4091-4932-43c9-924e-9f17470da231/discovery/0.log" Jul 22 04:19:56.936318 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:56.936280 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vrktd_5ab6787b-526f-438f-935b-8a400e7c4b68/istio-proxy/0.log" Jul 22 04:19:56.964817 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:56.964774 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vrktd_5ab6787b-526f-438f-935b-8a400e7c4b68/istio-proxy/1.log" Jul 22 04:19:57.013137 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:57.013024 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l4skc_95b1b413-599a-474c-8987-a7786d3c6817/payload-processing/0.log" Jul 22 04:19:58.287662 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:58.287637 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf47mgtz_3a5870d1-e07b-48ed-bd97-ba6406dc054c/istio-proxy/0.log" Jul 22 04:19:58.418099 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:58.418067 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-ck86j_d0af4091-4932-43c9-924e-9f17470da231/discovery/0.log" Jul 22 04:19:58.466747 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:58.466702 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vrktd_5ab6787b-526f-438f-935b-8a400e7c4b68/istio-proxy/0.log" Jul 22 04:19:58.492543 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:58.492518 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-56654dcf7-vrktd_5ab6787b-526f-438f-935b-8a400e7c4b68/istio-proxy/1.log" Jul 22 04:19:58.540134 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:19:58.540073 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-l4skc_95b1b413-599a-474c-8987-a7786d3c6817/payload-processing/0.log" Jul 22 04:20:01.273951 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:01.273918 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pmzq8_2d0f7975-a0a0-4ab5-936d-8025af234065/migrator/0.log" Jul 22 04:20:01.301625 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:01.301592 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7cc9c7ffd7-pmzq8_2d0f7975-a0a0-4ab5-936d-8025af234065/graceful-termination/0.log" Jul 22 04:20:02.904560 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:02.904533 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wddc_8f61a890-2b9e-46a7-b95d-2fb102461e5f/kube-multus-additional-cni-plugins/0.log" Jul 22 04:20:02.923897 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:02.923867 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wddc_8f61a890-2b9e-46a7-b95d-2fb102461e5f/egress-router-binary-copy/0.log" Jul 22 04:20:02.944140 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:02.944108 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wddc_8f61a890-2b9e-46a7-b95d-2fb102461e5f/cni-plugins/0.log" Jul 22 04:20:02.962579 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:02.962546 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wddc_8f61a890-2b9e-46a7-b95d-2fb102461e5f/bond-cni-plugin/0.log" Jul 22 04:20:02.981767 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:02.981734 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wddc_8f61a890-2b9e-46a7-b95d-2fb102461e5f/routeoverride-cni/0.log" Jul 22 04:20:03.001978 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:03.001948 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wddc_8f61a890-2b9e-46a7-b95d-2fb102461e5f/whereabouts-cni-bincopy/0.log" Jul 22 04:20:03.024123 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:03.024085 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7wddc_8f61a890-2b9e-46a7-b95d-2fb102461e5f/whereabouts-cni/0.log" Jul 22 04:20:03.217108 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:03.217017 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q5wp2_99146d59-9616-4c58-8989-c5b861c1b363/kube-multus/0.log" Jul 22 04:20:03.314392 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:03.314359 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m79rh_41c2a5d1-2687-40c4-b1dc-869e2f36ee58/network-metrics-daemon/0.log" Jul 22 04:20:03.332061 ip-10-0-139-52 kubenswrapper[2564]: I0722 04:20:03.332032 2564 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-m79rh_41c2a5d1-2687-40c4-b1dc-869e2f36ee58/kube-rbac-proxy/0.log"