Jul 21 17:08:19.069237 ip-10-0-142-109 systemd[1]: Starting Kubernetes Kubelet... Jul 21 17:08:20.785871 ip-10-0-142-109 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 17:08:20.785871 ip-10-0-142-109 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 21 17:08:20.785871 ip-10-0-142-109 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 17:08:20.785871 ip-10-0-142-109 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 21 17:08:20.785871 ip-10-0-142-109 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 21 17:08:20.786775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.786661 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 21 17:08:20.790209 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790184 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790212 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790217 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790221 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790224 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790228 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790231 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790250 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790255 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790259 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790263 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790268 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:08:20.790264 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790271 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790274 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790277 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790286 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790289 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790291 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790294 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790297 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790300 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790302 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790306 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790308 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790311 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790313 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790316 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790319 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790321 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790324 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790326 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790329 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:08:20.790567 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790331 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790334 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790336 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790339 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790341 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790343 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790346 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790349 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790351 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790354 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790356 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790360 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790363 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790372 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790375 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790378 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790380 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790383 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790386 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:08:20.791060 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790388 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790391 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790394 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790397 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790399 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790402 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790405 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790407 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790410 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790413 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790415 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790418 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790420 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790423 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790425 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790428 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790431 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790433 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790436 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790439 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:08:20.791573 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790442 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790444 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790447 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790449 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790454 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790456 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790459 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790464 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790468 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790471 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790474 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790477 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790479 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790482 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790485 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790487 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790489 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790492 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790494 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790497 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:08:20.792045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790500 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790502 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790504 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790507 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790509 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790513 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790516 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.790519 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791274 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791281 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791284 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791287 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791290 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791293 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791296 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791299 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791302 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791307 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791311 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:08:20.792575 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791315 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791318 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791321 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791324 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791327 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791329 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791332 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791336 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791339 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791343 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791346 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791348 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791351 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791353 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791356 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791359 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791363 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791366 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791369 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791371 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:08:20.793147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791375 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791378 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791380 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791383 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791385 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791391 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791394 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791396 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791399 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791402 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791405 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791408 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791411 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791414 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791417 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791420 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791423 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791425 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791428 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791431 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:08:20.793696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791434 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791437 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791439 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791441 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791444 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791447 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791450 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791452 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791455 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791457 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791460 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791462 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791465 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791468 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791471 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791474 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791476 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791479 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791483 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:08:20.794188 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791485 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791489 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791491 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791494 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791497 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791500 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791503 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791505 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791507 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791510 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791513 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791515 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791517 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791520 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791523 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791525 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791528 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791531 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791533 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791536 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:08:20.794669 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791539 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791541 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791544 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791546 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791549 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791551 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791554 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791557 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791559 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791939 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791944 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791947 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791950 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791953 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791956 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791959 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791961 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791964 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791966 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:08:20.795146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791969 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791972 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791974 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791976 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791979 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791982 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791985 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791987 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791989 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791992 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791995 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.791998 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792001 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792003 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792006 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792008 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792011 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792013 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792015 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792018 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:08:20.795610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792020 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792023 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792027 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792029 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792032 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792035 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792037 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792040 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792042 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792045 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792047 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792050 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792053 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792055 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792058 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792060 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792063 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792066 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792068 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792071 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792073 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:08:20.796080 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792076 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792079 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792082 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792085 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792087 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792089 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792092 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792094 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792097 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792099 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792102 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792104 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792107 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792110 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792112 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792115 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792117 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792120 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792123 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:08:20.796592 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792127 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792130 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792133 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792135 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792138 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792140 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792143 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792145 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792148 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792150 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792152 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792155 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792157 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792160 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792163 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792166 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792170 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792174 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792176 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:08:20.797057 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792180 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792183 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792186 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792189 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792191 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792194 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792197 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792200 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792203 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.792205 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792301 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792312 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792323 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792330 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792336 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792341 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792347 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792351 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792355 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792358 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792361 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 21 17:08:20.797525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792364 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792368 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792371 2565 flags.go:64] FLAG: --cgroup-root="" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792375 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792378 2565 flags.go:64] FLAG: --client-ca-file="" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792381 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792384 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792390 2565 flags.go:64] FLAG: --cluster-domain="" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792393 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792396 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792400 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792403 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792407 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792410 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792413 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792417 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792420 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792423 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792426 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792430 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792433 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792437 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792440 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792443 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792447 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 21 17:08:20.798018 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792450 2565 flags.go:64] FLAG: --enable-server="true" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792453 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792459 2565 flags.go:64] FLAG: --event-burst="100" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792462 2565 flags.go:64] FLAG: --event-qps="50" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792464 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792468 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792471 2565 flags.go:64] FLAG: --eviction-hard="" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792474 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792477 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792480 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792483 2565 flags.go:64] FLAG: --eviction-soft="" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792486 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792489 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792492 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792497 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792500 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792503 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792506 2565 flags.go:64] FLAG: --feature-gates="" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792510 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792513 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792516 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792519 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792522 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792525 2565 flags.go:64] FLAG: --help="false" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792528 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-142-109.ec2.internal" Jul 21 17:08:20.798627 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792531 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792534 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792537 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792541 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792544 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792547 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792550 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792557 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792560 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792563 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792566 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792569 2565 flags.go:64] FLAG: --kube-reserved="" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792572 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792575 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792578 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792581 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792584 2565 flags.go:64] FLAG: --lock-file="" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792587 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792590 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792593 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792601 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792605 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792608 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 21 17:08:20.799306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792611 2565 flags.go:64] FLAG: --logging-format="text" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792613 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792617 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792619 2565 flags.go:64] FLAG: --manifest-url="" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792622 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792627 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792631 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792635 2565 flags.go:64] FLAG: --max-pods="110" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792638 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792641 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792644 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792647 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792650 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792653 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.792656 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793047 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793065 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793070 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793074 2565 flags.go:64] FLAG: --pod-cidr="" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793078 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793086 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793089 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793093 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793097 2565 flags.go:64] FLAG: --port="10250" Jul 21 17:08:20.799947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793100 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793104 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-041cba80d81310208" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793107 2565 flags.go:64] FLAG: --qos-reserved="" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793111 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793114 2565 flags.go:64] FLAG: --register-node="true" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793117 2565 flags.go:64] FLAG: --register-with-taints="" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793123 2565 flags.go:64] FLAG: --registry-burst="10" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793126 2565 flags.go:64] FLAG: --registry-qps="5" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793129 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793132 2565 flags.go:64] FLAG: --reserved-memory="" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793136 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793139 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793143 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793146 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793149 2565 flags.go:64] FLAG: --runonce="false" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793152 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793155 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793159 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793162 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793165 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793169 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793172 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793175 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793178 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793183 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 21 17:08:20.800911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793189 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793192 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793196 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793199 2565 flags.go:64] FLAG: --system-cgroups="" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793202 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793211 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793214 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793217 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793222 2565 flags.go:64] FLAG: --tls-min-version="" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793225 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793228 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793231 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793248 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793252 2565 flags.go:64] FLAG: --v="2" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793257 2565 flags.go:64] FLAG: --version="false" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793261 2565 flags.go:64] FLAG: --vmodule="" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793265 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793269 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793388 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793392 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793395 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793397 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793401 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793404 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:08:20.801661 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793406 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793409 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793411 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793414 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793418 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793423 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793426 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793430 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793435 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793438 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793441 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793444 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793447 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793449 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793452 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793455 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793458 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793460 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793463 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:08:20.802327 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793465 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793468 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793470 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793473 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793475 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793478 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793481 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793483 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793486 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793489 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793493 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793497 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793500 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793502 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793505 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793507 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793510 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793513 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793515 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:08:20.802788 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793517 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793520 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793524 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793528 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793530 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793533 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793535 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793538 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793541 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793544 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793546 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793548 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793551 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793553 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793556 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793559 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793561 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793564 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793566 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793569 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:08:20.803300 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793572 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793574 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793576 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793579 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793581 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793589 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793592 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793594 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793597 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793599 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793602 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793605 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793607 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793610 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793614 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793616 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793619 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793622 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793625 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793627 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:08:20.803777 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793630 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793633 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793635 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793638 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793640 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793643 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793645 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793648 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793650 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793653 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793655 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793658 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793661 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793663 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.793666 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:08:20.804285 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.793671 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.802094 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.802118 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.802783 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.802796 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802865 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802871 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802874 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802878 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802881 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802884 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802887 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802890 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802893 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802895 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802898 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802901 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802904 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802907 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802909 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:08:20.804667 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802912 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802914 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802917 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802920 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802924 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802928 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802931 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802933 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802937 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802940 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802943 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802945 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802948 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802950 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802954 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802956 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802959 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802961 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802964 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:08:20.805147 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802967 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802969 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802972 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802974 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802977 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802979 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802982 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802984 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802988 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802991 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802994 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.802997 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803000 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803002 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803005 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803008 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803011 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803013 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803016 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:08:20.805628 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803019 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803021 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803024 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803027 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803029 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803031 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803034 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803036 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803040 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803043 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803045 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803048 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803051 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803053 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803056 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803058 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803061 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803065 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803068 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803071 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:08:20.806079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803074 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803077 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803080 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803082 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803085 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803087 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803091 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803093 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803096 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803099 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803101 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803104 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803107 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803109 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803112 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803114 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803117 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803119 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803122 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803125 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:08:20.806595 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803128 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803131 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803134 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803136 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803139 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803142 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.803147 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803292 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803299 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803302 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803305 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803308 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803311 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803314 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 21 17:08:20.807084 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803318 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803321 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803323 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803326 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803329 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803332 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803335 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803337 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803340 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803342 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803345 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803348 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803350 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803353 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803355 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803358 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803360 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803363 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803365 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803370 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 21 17:08:20.807449 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803374 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803377 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803380 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803382 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803385 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803387 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803390 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803393 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803395 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803397 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803400 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803403 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803407 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803410 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803413 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803416 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803419 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803422 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803425 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 21 17:08:20.807917 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803428 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803431 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803433 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803436 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803438 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803440 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803443 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803446 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803448 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803451 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803453 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803456 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803459 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803462 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803464 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803467 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803469 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803472 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803475 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803477 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803480 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 21 17:08:20.808385 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803482 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803485 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803487 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803490 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803492 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803495 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803497 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803500 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803503 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803505 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803507 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803515 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803518 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803521 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803523 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803526 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803528 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803531 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803533 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 21 17:08:20.808880 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803536 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803540 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803543 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803547 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803550 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803553 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803556 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803559 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803562 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803565 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803568 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803570 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:20.803573 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.803578 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 21 17:08:20.809501 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.803715 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 21 17:08:20.809840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.807097 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 21 17:08:20.809840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.808397 2565 server.go:1019] "Starting client certificate rotation" Jul 21 17:08:20.809840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.808504 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 17:08:20.809840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.808547 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 21 17:08:20.810545 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.810530 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 17:08:20.811801 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.811770 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 21 17:08:20.819512 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.819492 2565 log.go:25] "Validated CRI v1 runtime API" Jul 21 17:08:20.821748 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.821729 2565 log.go:25] "Validated CRI v1 image API" Jul 21 17:08:20.822638 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.822615 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 21 17:08:20.825085 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.825065 2565 fs.go:135] Filesystem UUIDs: map[63b1e327-5072-4f91-aca8-6f1c7c13173f:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 f88d0533-3b9a-4635-bdfd-cbc4b75d7585:/dev/nvme0n1p3] Jul 21 17:08:20.825150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.825085 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 21 17:08:20.831635 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.831503 2565 manager.go:216] Machine: {Timestamp:2026-07-21 17:08:20.830412517 +0000 UTC m=+0.443956808 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101192 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2099f9fffed7a0aa11d043fdfe8351 SystemUUID:ec2099f9-fffe-d7a0-aa11-d043fdfe8351 BootID:7f88514c-f0f9-47cc-8029-d62db26f5bf6 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:74:92:ef:3d:ab Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:74:92:ef:3d:ab Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:71:0e:66:e9:50 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 21 17:08:20.831635 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.831618 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 21 17:08:20.831814 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.831738 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 21 17:08:20.834278 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.834250 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 21 17:08:20.834460 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.834283 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-109.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 21 17:08:20.834932 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.834920 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 21 17:08:20.834992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.834938 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 21 17:08:20.834992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.834949 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 17:08:20.835516 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.835504 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 21 17:08:20.836711 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.836698 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 21 17:08:20.836996 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.836978 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 17:08:20.840348 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840331 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 21 17:08:20.840500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840489 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 21 17:08:20.840569 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840528 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 21 17:08:20.840569 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840541 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 21 17:08:20.840569 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840563 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 21 17:08:20.840680 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840579 2565 kubelet.go:398] "Adding apiserver pod source" Jul 21 17:08:20.840680 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840593 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 21 17:08:20.841008 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.840995 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 17:08:20.841080 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.841018 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 21 17:08:20.842889 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.842874 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 21 17:08:20.844616 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.844603 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 21 17:08:20.844661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.844633 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 21 17:08:20.846170 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846159 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 21 17:08:20.846215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846175 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 21 17:08:20.846215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846181 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 21 17:08:20.846215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846187 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 21 17:08:20.846215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846205 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 21 17:08:20.846215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846213 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 21 17:08:20.846381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846222 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 21 17:08:20.846381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846228 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 21 17:08:20.846381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846249 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 21 17:08:20.846381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846278 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 21 17:08:20.846381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846296 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 21 17:08:20.846381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.846303 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 21 17:08:20.847085 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.847070 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 21 17:08:20.847126 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.847090 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 21 17:08:20.850646 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.850631 2565 server.go:1268] "Started kubelet" Jul 21 17:08:20.850887 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.850855 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 21 17:08:20.851202 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.851140 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 21 17:08:20.851299 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.851222 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 21 17:08:20.851625 ip-10-0-142-109 systemd[1]: Started Kubernetes Kubelet. Jul 21 17:08:20.853005 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.852990 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 21 17:08:20.853341 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.853306 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 21 17:08:20.853825 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.853601 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 21 17:08:20.853825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.853633 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-109.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 21 17:08:20.853825 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.853644 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-109.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 21 17:08:20.855904 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.855049 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-109.ec2.internal.18c45d15de4941d0 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-109.ec2.internal,UID:ip-10-0-142-109.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-109.ec2.internal,},FirstTimestamp:2026-07-21 17:08:20.850598352 +0000 UTC m=+0.464142644,LastTimestamp:2026-07-21 17:08:20.850598352 +0000 UTC m=+0.464142644,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-109.ec2.internal,}" Jul 21 17:08:20.856073 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.856052 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 21 17:08:20.859472 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.859453 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 21 17:08:20.859472 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.859465 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 21 17:08:20.859603 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.859554 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:20.859985 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.859972 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 21 17:08:20.860090 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.859974 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 21 17:08:20.860182 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860093 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 21 17:08:20.860297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860220 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 21 17:08:20.860297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860230 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 21 17:08:20.860673 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860656 2565 factory.go:55] Registering systemd factory Jul 21 17:08:20.860762 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860678 2565 factory.go:223] Registration of the systemd container factory successfully Jul 21 17:08:20.860885 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860867 2565 factory.go:153] Registering CRI-O factory Jul 21 17:08:20.860885 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860884 2565 factory.go:223] Registration of the crio container factory successfully Jul 21 17:08:20.861019 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860936 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 21 17:08:20.861019 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860955 2565 factory.go:103] Registering Raw factory Jul 21 17:08:20.861019 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.860969 2565 manager.go:1196] Started watching for new ooms in manager Jul 21 17:08:20.861981 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.861968 2565 manager.go:349] Starting recovery of all containers Jul 21 17:08:20.862582 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.862555 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 21 17:08:20.862582 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.862541 2565 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-109.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 21 17:08:20.868216 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.865382 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s9fv2" Jul 21 17:08:20.871336 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.871307 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-s9fv2" Jul 21 17:08:20.875310 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.875296 2565 manager.go:354] Recovery completed Jul 21 17:08:20.876697 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.876669 2565 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 21 17:08:20.879689 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.879676 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:08:20.881731 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.881712 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:08:20.881803 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.881748 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:08:20.881803 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.881764 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:08:20.882955 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.882933 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 21 17:08:20.882955 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.882949 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:08:20.883096 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.882951 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 21 17:08:20.883096 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.883045 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 21 17:08:20.883799 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.883773 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:08:20.883887 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.883808 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:08:20.883887 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.883824 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:08:20.884179 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.884104 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-109.ec2.internal.18c45d15e0244d07 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-109.ec2.internal,UID:ip-10-0-142-109.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-109.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-109.ec2.internal,},FirstTimestamp:2026-07-21 17:08:20.881730823 +0000 UTC m=+0.495275118,LastTimestamp:2026-07-21 17:08:20.881730823 +0000 UTC m=+0.495275118,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-109.ec2.internal,}" Jul 21 17:08:20.885054 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.885039 2565 policy_none.go:49] "None policy: Start" Jul 21 17:08:20.885054 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.885056 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 21 17:08:20.885153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.885066 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 21 17:08:20.887027 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.887017 2565 policy_none.go:47] "Start" Jul 21 17:08:20.925091 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.924892 2565 manager.go:337] "Starting Device Plugin manager" Jul 21 17:08:20.925231 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.925130 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 21 17:08:20.925231 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.925145 2565 server.go:87] "Starting device plugin registration server" Jul 21 17:08:20.925477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.925462 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 21 17:08:20.925545 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.925479 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 21 17:08:20.925594 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.925580 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 21 17:08:20.925671 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.925658 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 21 17:08:20.925671 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:20.925669 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 21 17:08:20.926955 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.926929 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 21 17:08:20.927087 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:20.926990 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.024665 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.024627 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.025835 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.025901 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.025940 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.025962 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.025998 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.026628 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.026658 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.026674 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:08:21.026907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.026704 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.029821 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.029802 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 17:08:21.036416 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.036388 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.036416 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.036417 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-109.ec2.internal\": node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.105572 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.105484 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.126602 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.126570 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal"] Jul 21 17:08:21.126681 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.126659 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:08:21.128153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.128136 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:08:21.128281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.128166 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:08:21.128281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.128175 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:08:21.130489 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.130474 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:08:21.130629 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.130593 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.130680 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.130630 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:08:21.131271 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.131256 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:08:21.131362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.131257 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:08:21.131362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.131287 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:08:21.131362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.131301 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:08:21.131362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.131305 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:08:21.131362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.131316 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:08:21.134188 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.134174 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.134283 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.134209 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 21 17:08:21.134897 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.134879 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientMemory" Jul 21 17:08:21.134984 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.134908 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasNoDiskPressure" Jul 21 17:08:21.134984 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.134918 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeHasSufficientPID" Jul 21 17:08:21.157133 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.157109 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-109.ec2.internal\" not found" node="ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.161648 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.161624 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ee6ac37bca8309a5190f95f0c1bc299-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal\" (UID: \"9ee6ac37bca8309a5190f95f0c1bc299\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.161777 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.161653 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ee6ac37bca8309a5190f95f0c1bc299-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal\" (UID: \"9ee6ac37bca8309a5190f95f0c1bc299\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.161777 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.161676 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70278ed5d98e32abf177b8a325b5bf17-config\") pod \"kube-apiserver-proxy-ip-10-0-142-109.ec2.internal\" (UID: \"70278ed5d98e32abf177b8a325b5bf17\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.161777 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.161710 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-109.ec2.internal\" not found" node="ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.206525 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.206491 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.262509 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.262471 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ee6ac37bca8309a5190f95f0c1bc299-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal\" (UID: \"9ee6ac37bca8309a5190f95f0c1bc299\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.262509 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.262506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ee6ac37bca8309a5190f95f0c1bc299-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal\" (UID: \"9ee6ac37bca8309a5190f95f0c1bc299\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.262699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.262530 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70278ed5d98e32abf177b8a325b5bf17-config\") pod \"kube-apiserver-proxy-ip-10-0-142-109.ec2.internal\" (UID: \"70278ed5d98e32abf177b8a325b5bf17\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.262699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.262559 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9ee6ac37bca8309a5190f95f0c1bc299-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal\" (UID: \"9ee6ac37bca8309a5190f95f0c1bc299\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.262699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.262575 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ee6ac37bca8309a5190f95f0c1bc299-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal\" (UID: \"9ee6ac37bca8309a5190f95f0c1bc299\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.262699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.262584 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/70278ed5d98e32abf177b8a325b5bf17-config\") pod \"kube-apiserver-proxy-ip-10-0-142-109.ec2.internal\" (UID: \"70278ed5d98e32abf177b8a325b5bf17\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.306921 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.306880 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.407557 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.407475 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.460717 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.460676 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.463830 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.463810 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" Jul 21 17:08:21.508061 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.508025 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.608547 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.608505 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.709048 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.708967 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.808468 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.808435 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 21 17:08:21.808984 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.808620 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 21 17:08:21.809534 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.809512 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:21.876970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.876938 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-20 17:03:20 +0000 UTC" deadline="2028-04-16 07:25:06.850153362 +0000 UTC" Jul 21 17:08:21.876970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.876968 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15230h16m44.973188715s" Jul 21 17:08:21.882283 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:21.882260 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 17:08:21.910676 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:21.910645 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:22.002146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:22.002105 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee6ac37bca8309a5190f95f0c1bc299.slice/crio-b2eaefa2fa4806c49a860cca6e53c9814dc3df6a7ff03830713321aa636421e1 WatchSource:0}: Error finding container b2eaefa2fa4806c49a860cca6e53c9814dc3df6a7ff03830713321aa636421e1: Status 404 returned error can't find the container with id b2eaefa2fa4806c49a860cca6e53c9814dc3df6a7ff03830713321aa636421e1 Jul 21 17:08:22.002696 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:22.002667 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70278ed5d98e32abf177b8a325b5bf17.slice/crio-3afddf2d41c8f1c39ee011a42719cbf79985f53aaf1e649c3e2436b77e301bf6 WatchSource:0}: Error finding container 3afddf2d41c8f1c39ee011a42719cbf79985f53aaf1e649c3e2436b77e301bf6: Status 404 returned error can't find the container with id 3afddf2d41c8f1c39ee011a42719cbf79985f53aaf1e649c3e2436b77e301bf6 Jul 21 17:08:22.006748 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.006729 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 17:08:22.011440 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.011421 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-109.ec2.internal\" not found" Jul 21 17:08:22.013945 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.013929 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 17:08:22.029400 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.029352 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" event={"ID":"9ee6ac37bca8309a5190f95f0c1bc299","Type":"ContainerStarted","Data":"b2eaefa2fa4806c49a860cca6e53c9814dc3df6a7ff03830713321aa636421e1"} Jul 21 17:08:22.030293 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.030264 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" event={"ID":"70278ed5d98e32abf177b8a325b5bf17","Type":"ContainerStarted","Data":"3afddf2d41c8f1c39ee011a42719cbf79985f53aaf1e649c3e2436b77e301bf6"} Jul 21 17:08:22.060536 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.060503 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" Jul 21 17:08:22.072344 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.072320 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 17:08:22.073209 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.073196 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" Jul 21 17:08:22.081335 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.081309 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 21 17:08:22.331494 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.331400 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 17:08:22.841811 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.841635 2565 apiserver.go:52] "Watching apiserver" Jul 21 17:08:22.849581 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.849557 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 21 17:08:22.850883 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.850854 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-mkhk5","openshift-multus/multus-additional-cni-plugins-xhljt","openshift-multus/multus-wjfzj","openshift-multus/network-metrics-daemon-xrl6x","openshift-network-diagnostics/network-check-target-26bm7","openshift-network-operator/iptables-alerter-r99rt","kube-system/global-pull-secret-syncer-slspc","kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal","openshift-dns/node-resolver-5vmr7","openshift-image-registry/node-ca-wqfb5","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal","openshift-ovn-kubernetes/ovnkube-node-q45lm","kube-system/konnectivity-agent-7f57l","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj"] Jul 21 17:08:22.853481 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.853455 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.853613 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.853545 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:22.855635 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.855614 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:22.857886 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.857861 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.858292 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.858272 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 21 17:08:22.858397 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.858326 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-hmphm\"" Jul 21 17:08:22.858458 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.858274 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 21 17:08:22.859821 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.859798 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 21 17:08:22.860458 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.860438 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 21 17:08:22.860559 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.860490 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 21 17:08:22.860711 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.860638 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 21 17:08:22.860776 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.860700 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:22.861282 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.861211 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:22.861974 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.861231 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 21 17:08:22.861974 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.861125 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pr7l4\"" Jul 21 17:08:22.864197 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.864174 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:22.864301 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.864268 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:22.866615 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.866593 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:22.868866 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.868842 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:22.869453 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.869431 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 21 17:08:22.869453 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.869444 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 21 17:08:22.869594 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.869495 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t945v\"" Jul 21 17:08:22.869594 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.869431 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 21 17:08:22.869938 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.869915 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-dbus\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.870003 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.869953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.870003 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.869979 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-cni-bin\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870079 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870003 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-multus-certs\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870079 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870027 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-etc-kubernetes\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870079 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:22.870254 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870115 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4d06660-fbc8-4804-96be-69563b12276d-cni-binary-copy\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870254 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870150 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:22.870254 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870179 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-kubelet-config\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.870254 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-system-cni-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870255 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-cni-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870279 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-cnibin\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870299 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-os-release\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870321 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-k8s-cni-cncf-io\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870362 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b818316-6b2d-4e37-b96e-ff47692a0fd9-agent-certs\") pod \"konnectivity-agent-7f57l\" (UID: \"8b818316-6b2d-4e37-b96e-ff47692a0fd9\") " pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:22.870445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870400 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-socket-dir-parent\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870430 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-cni-multus\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-kubelet\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870493 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-hostroot\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870524 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-conf-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4d06660-fbc8-4804-96be-69563b12276d-multus-daemon-config\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870584 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxtd4\" (UniqueName: \"kubernetes.io/projected/b4d06660-fbc8-4804-96be-69563b12276d-kube-api-access-lxtd4\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870638 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b818316-6b2d-4e37-b96e-ff47692a0fd9-konnectivity-ca\") pod \"konnectivity-agent-7f57l\" (UID: \"8b818316-6b2d-4e37-b96e-ff47692a0fd9\") " pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-netns\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.870771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.870696 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6svgt\" (UniqueName: \"kubernetes.io/projected/17870260-a36e-451c-93ac-56c19cdb45c7-kube-api-access-6svgt\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:22.871166 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.871148 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-fdmt2\"" Jul 21 17:08:22.871222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.871164 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 21 17:08:22.871222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.871186 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 21 17:08:22.871433 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.871417 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.873795 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.873777 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.873885 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.873826 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-h4xbd\"" Jul 21 17:08:22.873951 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.873891 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 21 17:08:22.874004 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.873828 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 21 17:08:22.875836 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.875819 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 21 17:08:22.876057 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.876041 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:22.876424 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.876289 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-6x82w\"" Jul 21 17:08:22.876424 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.876301 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 21 17:08:22.877731 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.877710 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 21 17:08:22.878366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.878346 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.879862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.879838 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-r5v5j\"" Jul 21 17:08:22.879960 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.879938 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 21 17:08:22.880386 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.880259 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 21 17:08:22.880481 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.880442 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 21 17:08:22.880958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.880805 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 21 17:08:22.880958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.880903 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qrsq8\"" Jul 21 17:08:22.880958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.880911 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 21 17:08:22.880958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.880955 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.881200 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.881072 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 21 17:08:22.883713 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.883686 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-ksf6w\"" Jul 21 17:08:22.883958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.883944 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 21 17:08:22.884651 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.884629 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 21 17:08:22.884756 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.884690 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 21 17:08:22.884756 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.884636 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 21 17:08:22.884939 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.884636 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 21 17:08:22.885014 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.884945 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 21 17:08:22.902219 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.902196 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xdxn" Jul 21 17:08:22.910930 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.910906 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-2xdxn" Jul 21 17:08:22.961345 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.961299 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 21 17:08:22.971409 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-netns\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971421 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9246p\" (UniqueName: \"kubernetes.io/projected/93b92f78-4af1-41c9-9464-fa3b32effaa6-kube-api-access-9246p\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:22.971591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971503 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.971591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-registration-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.971591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971550 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-netns\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971565 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-run-netns\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.971591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-cni-netd\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971628 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-etc-kubernetes\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971686 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-etc-kubernetes\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-system-cni-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-os-release\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-kubernetes\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971778 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47e09fe0-e9a5-43d9-98db-4a6d131eff09-tmp\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-kubelet\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971813 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxtd4\" (UniqueName: \"kubernetes.io/projected/b4d06660-fbc8-4804-96be-69563b12276d-kube-api-access-lxtd4\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.971857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-systemd\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971876 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-sys\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971894 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-device-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-etc-selinux\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971935 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d49b95fd-0f47-423f-a36a-536f9ef79bf9-serviceca\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971950 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-etc-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971971 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.971987 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-ovnkube-script-lib\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-dbus\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972055 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-cni-bin\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-multus-certs\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972105 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-system-cni-dir\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972122 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972145 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d49b95fd-0f47-423f-a36a-536f9ef79bf9-host\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972160 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-ovn\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.972173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972174 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-log-socket\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972189 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-conf-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-kubelet-config\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972223 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-cni-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972265 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-cnibin\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972288 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972306 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-kubelet\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972344 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-os-release\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972363 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-cni-bin\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972386 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-conf-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-system-cni-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972402 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-dbus\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972419 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-cni-dir\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-run\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972461 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-cnibin\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972436 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-multus-certs\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972468 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-lib-modules\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972509 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-ovnkube-config\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.972826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972546 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-kubelet-config\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972581 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4d06660-fbc8-4804-96be-69563b12276d-multus-daemon-config\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93b92f78-4af1-41c9-9464-fa3b32effaa6-iptables-alerter-script\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972753 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93b92f78-4af1-41c9-9464-fa3b32effaa6-host-slash\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972796 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-modprobe-d\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972824 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-tuned\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972846 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-slash\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972887 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972912 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-env-overrides\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.972972 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b818316-6b2d-4e37-b96e-ff47692a0fd9-konnectivity-ca\") pod \"konnectivity-agent-7f57l\" (UID: \"8b818316-6b2d-4e37-b96e-ff47692a0fd9\") " pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973017 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973055 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-node-log\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973083 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-run-ovn-kubernetes\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-cni-bin\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58c69f2f-02ff-4043-a868-aca57a713daa-ovn-node-metrics-cert\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973159 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvf8b\" (UniqueName: \"kubernetes.io/projected/58c69f2f-02ff-4043-a868-aca57a713daa-kube-api-access-tvf8b\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.973178 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:22.973744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973215 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b4d06660-fbc8-4804-96be-69563b12276d-multus-daemon-config\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.973288 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs podName:17870260-a36e-451c-93ac-56c19cdb45c7 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:23.473266703 +0000 UTC m=+3.086811002 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs") pod "network-metrics-daemon-xrl6x" (UID: "17870260-a36e-451c-93ac-56c19cdb45c7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973184 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4d06660-fbc8-4804-96be-69563b12276d-cni-binary-copy\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973325 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-k8s-cni-cncf-io\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973353 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973379 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysctl-d\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973399 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-host\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973422 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b818316-6b2d-4e37-b96e-ff47692a0fd9-agent-certs\") pod \"konnectivity-agent-7f57l\" (UID: \"8b818316-6b2d-4e37-b96e-ff47692a0fd9\") " pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973444 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-hostroot\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973465 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-cnibin\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973489 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973515 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chwxw\" (UniqueName: \"kubernetes.io/projected/d62be561-533b-4be0-8ab5-6ea408150db5-kube-api-access-chwxw\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973527 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-run-k8s-cni-cncf-io\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973538 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-sys-fs\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973559 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-systemd\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973581 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-hostroot\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.974464 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973587 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/8b818316-6b2d-4e37-b96e-ff47692a0fd9-konnectivity-ca\") pod \"konnectivity-agent-7f57l\" (UID: \"8b818316-6b2d-4e37-b96e-ff47692a0fd9\") " pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973626 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svgt\" (UniqueName: \"kubernetes.io/projected/17870260-a36e-451c-93ac-56c19cdb45c7-kube-api-access-6svgt\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973660 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b4d06660-fbc8-4804-96be-69563b12276d-cni-binary-copy\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973671 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq9wc\" (UniqueName: \"kubernetes.io/projected/163f8c01-98b8-4a45-b3b5-461b057c4349-kube-api-access-sq9wc\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973695 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysconfig\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973749 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8drc8\" (UniqueName: \"kubernetes.io/projected/d49b95fd-0f47-423f-a36a-536f9ef79bf9-kube-api-access-8drc8\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973786 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973814 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nzh7\" (UniqueName: \"kubernetes.io/projected/013607f3-00b2-48cb-9e89-d8a3cddafd22-kube-api-access-9nzh7\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-var-lib-kubelet\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973862 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973915 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-socket-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973945 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/013607f3-00b2-48cb-9e89-d8a3cddafd22-hosts-file\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973968 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-systemd-units\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.973991 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-var-lib-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-socket-dir-parent\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974046 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-cni-multus\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-os-release\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:22.975068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/013607f3-00b2-48cb-9e89-d8a3cddafd22-tmp-dir\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:22.975656 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974110 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysctl-conf\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.975656 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974121 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-host-var-lib-cni-multus\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.975656 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974133 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l26mr\" (UniqueName: \"kubernetes.io/projected/47e09fe0-e9a5-43d9-98db-4a6d131eff09-kube-api-access-l26mr\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:22.975656 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.973889 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:22.975656 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974161 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-kubelet\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:22.975656 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.974216 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b4d06660-fbc8-4804-96be-69563b12276d-multus-socket-dir-parent\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.975656 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.974226 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret podName:ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:23.474204016 +0000 UTC m=+3.087748299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret") pod "global-pull-secret-syncer-slspc" (UID: "ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4") : object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:22.977314 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.977295 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/8b818316-6b2d-4e37-b96e-ff47692a0fd9-agent-certs\") pod \"konnectivity-agent-7f57l\" (UID: \"8b818316-6b2d-4e37-b96e-ff47692a0fd9\") " pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:22.987582 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.987543 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 17:08:22.987582 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.987574 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 17:08:22.987582 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.987590 2565 projected.go:196] Error preparing data for projected volume kube-api-access-5qqdt for pod openshift-network-diagnostics/network-check-target-26bm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:22.987888 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:22.987661 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt podName:decf50e7-9d50-402a-b7cd-e1c5589772b9 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:23.487642958 +0000 UTC m=+3.101187239 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqdt" (UniqueName: "kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt") pod "network-check-target-26bm7" (UID: "decf50e7-9d50-402a-b7cd-e1c5589772b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:22.989109 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.989087 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxtd4\" (UniqueName: \"kubernetes.io/projected/b4d06660-fbc8-4804-96be-69563b12276d-kube-api-access-lxtd4\") pod \"multus-wjfzj\" (UID: \"b4d06660-fbc8-4804-96be-69563b12276d\") " pod="openshift-multus/multus-wjfzj" Jul 21 17:08:22.989255 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:22.989137 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6svgt\" (UniqueName: \"kubernetes.io/projected/17870260-a36e-451c-93ac-56c19cdb45c7-kube-api-access-6svgt\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:23.059088 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.059049 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 21 17:08:23.075053 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075013 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-cnibin\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.075053 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chwxw\" (UniqueName: \"kubernetes.io/projected/d62be561-533b-4be0-8ab5-6ea408150db5-kube-api-access-chwxw\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075140 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-cnibin\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075156 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-sys-fs\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075173 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-systemd\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075189 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq9wc\" (UniqueName: \"kubernetes.io/projected/163f8c01-98b8-4a45-b3b5-461b057c4349-kube-api-access-sq9wc\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075228 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysconfig\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075229 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-sys-fs\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075280 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-systemd\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.075304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075309 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysconfig\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8drc8\" (UniqueName: \"kubernetes.io/projected/d49b95fd-0f47-423f-a36a-536f9ef79bf9-kube-api-access-8drc8\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9nzh7\" (UniqueName: \"kubernetes.io/projected/013607f3-00b2-48cb-9e89-d8a3cddafd22-kube-api-access-9nzh7\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075439 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075470 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-var-lib-kubelet\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075514 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-var-lib-kubelet\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-socket-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/013607f3-00b2-48cb-9e89-d8a3cddafd22-hosts-file\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-systemd-units\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075659 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-var-lib-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075688 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-os-release\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/013607f3-00b2-48cb-9e89-d8a3cddafd22-tmp-dir\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:23.075730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075739 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysctl-conf\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075746 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-socket-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075755 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-systemd-units\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l26mr\" (UniqueName: \"kubernetes.io/projected/47e09fe0-e9a5-43d9-98db-4a6d131eff09-kube-api-access-l26mr\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075799 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-kubelet\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075811 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/013607f3-00b2-48cb-9e89-d8a3cddafd22-hosts-file\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075819 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-os-release\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9246p\" (UniqueName: \"kubernetes.io/projected/93b92f78-4af1-41c9-9464-fa3b32effaa6-kube-api-access-9246p\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075849 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-var-lib-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075856 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-registration-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-run-netns\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075928 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-cni-netd\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075968 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-kubernetes\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.075992 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47e09fe0-e9a5-43d9-98db-4a6d131eff09-tmp\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076035 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-kubelet\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076064 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-run-netns\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.076291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076066 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076112 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-registration-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076119 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-cni-netd\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076128 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysctl-conf\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076117 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/013607f3-00b2-48cb-9e89-d8a3cddafd22-tmp-dir\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-systemd\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076173 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-kubernetes\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076177 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-sys\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076214 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-sys\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-device-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076225 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-systemd\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076265 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-etc-selinux\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076272 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-device-dir\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d49b95fd-0f47-423f-a36a-536f9ef79bf9-serviceca\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-etc-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076344 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076370 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-ovnkube-script-lib\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076381 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-etc-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076384 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/163f8c01-98b8-4a45-b3b5-461b057c4349-etc-selinux\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076400 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-system-cni-dir\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076450 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d49b95fd-0f47-423f-a36a-536f9ef79bf9-host\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076487 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-ovn\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076489 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d62be561-533b-4be0-8ab5-6ea408150db5-system-cni-dir\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076519 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-log-socket\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076538 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-ovn\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076551 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076579 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-run\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-lib-modules\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076650 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-ovnkube-config\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076676 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93b92f78-4af1-41c9-9464-fa3b32effaa6-iptables-alerter-script\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93b92f78-4af1-41c9-9464-fa3b32effaa6-host-slash\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-modprobe-d\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076761 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-tuned\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.077632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076781 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d49b95fd-0f47-423f-a36a-536f9ef79bf9-serviceca\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076825 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-log-socket\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076837 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-slash\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076882 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-run\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-slash\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076920 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076939 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-lib-modules\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076947 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-env-overrides\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076955 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-ovnkube-script-lib\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-node-log\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077009 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/93b92f78-4af1-41c9-9464-fa3b32effaa6-host-slash\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077035 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-run-openvswitch\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077044 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-cni-binary-copy\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077097 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d49b95fd-0f47-423f-a36a-536f9ef79bf9-host\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077103 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-modprobe-d\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.076448 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077131 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-run-ovn-kubernetes\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077150 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-cni-bin\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.078404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077167 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58c69f2f-02ff-4043-a868-aca57a713daa-ovn-node-metrics-cert\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077182 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvf8b\" (UniqueName: \"kubernetes.io/projected/58c69f2f-02ff-4043-a868-aca57a713daa-kube-api-access-tvf8b\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077207 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077225 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysctl-d\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077263 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-host\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077346 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-host\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077416 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-env-overrides\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077444 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-node-log\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077488 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-run-ovn-kubernetes\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077528 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/58c69f2f-02ff-4043-a868-aca57a713daa-host-cni-bin\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077704 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/58c69f2f-02ff-4043-a868-aca57a713daa-ovnkube-config\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077807 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.077924 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-sysctl-d\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.078008 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d62be561-533b-4be0-8ab5-6ea408150db5-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.078451 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/93b92f78-4af1-41c9-9464-fa3b32effaa6-iptables-alerter-script\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.078592 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/47e09fe0-e9a5-43d9-98db-4a6d131eff09-tmp\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.079306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.079303 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/47e09fe0-e9a5-43d9-98db-4a6d131eff09-etc-tuned\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.080117 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.080072 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/58c69f2f-02ff-4043-a868-aca57a713daa-ovn-node-metrics-cert\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.093053 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.092975 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l26mr\" (UniqueName: \"kubernetes.io/projected/47e09fe0-e9a5-43d9-98db-4a6d131eff09-kube-api-access-l26mr\") pod \"tuned-mkhk5\" (UID: \"47e09fe0-e9a5-43d9-98db-4a6d131eff09\") " pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.093261 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.093221 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nzh7\" (UniqueName: \"kubernetes.io/projected/013607f3-00b2-48cb-9e89-d8a3cddafd22-kube-api-access-9nzh7\") pod \"node-resolver-5vmr7\" (UID: \"013607f3-00b2-48cb-9e89-d8a3cddafd22\") " pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:23.093993 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.093971 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq9wc\" (UniqueName: \"kubernetes.io/projected/163f8c01-98b8-4a45-b3b5-461b057c4349-kube-api-access-sq9wc\") pod \"aws-ebs-csi-driver-node-tlwfj\" (UID: \"163f8c01-98b8-4a45-b3b5-461b057c4349\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.094256 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.094219 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8drc8\" (UniqueName: \"kubernetes.io/projected/d49b95fd-0f47-423f-a36a-536f9ef79bf9-kube-api-access-8drc8\") pod \"node-ca-wqfb5\" (UID: \"d49b95fd-0f47-423f-a36a-536f9ef79bf9\") " pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:23.097391 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.097357 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9246p\" (UniqueName: \"kubernetes.io/projected/93b92f78-4af1-41c9-9464-fa3b32effaa6-kube-api-access-9246p\") pod \"iptables-alerter-r99rt\" (UID: \"93b92f78-4af1-41c9-9464-fa3b32effaa6\") " pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:23.097766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.097749 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvf8b\" (UniqueName: \"kubernetes.io/projected/58c69f2f-02ff-4043-a868-aca57a713daa-kube-api-access-tvf8b\") pod \"ovnkube-node-q45lm\" (UID: \"58c69f2f-02ff-4043-a868-aca57a713daa\") " pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.102386 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.102361 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chwxw\" (UniqueName: \"kubernetes.io/projected/d62be561-533b-4be0-8ab5-6ea408150db5-kube-api-access-chwxw\") pod \"multus-additional-cni-plugins-xhljt\" (UID: \"d62be561-533b-4be0-8ab5-6ea408150db5\") " pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.171463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.171415 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:23.179635 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.179605 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wjfzj" Jul 21 17:08:23.189192 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.189162 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r99rt" Jul 21 17:08:23.195077 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.195046 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5vmr7" Jul 21 17:08:23.202037 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.202016 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" Jul 21 17:08:23.208826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.208795 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xhljt" Jul 21 17:08:23.216390 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.216365 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-wqfb5" Jul 21 17:08:23.223155 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.223132 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" Jul 21 17:08:23.228939 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.228916 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:23.479140 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.479043 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:23.479140 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.479096 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:23.479392 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.479203 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:23.479392 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.479204 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:23.479392 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.479273 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret podName:ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:24.479254473 +0000 UTC m=+4.092798753 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret") pod "global-pull-secret-syncer-slspc" (UID: "ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4") : object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:23.479392 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.479336 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs podName:17870260-a36e-451c-93ac-56c19cdb45c7 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:24.47931891 +0000 UTC m=+4.092863188 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs") pod "network-metrics-daemon-xrl6x" (UID: "17870260-a36e-451c-93ac-56c19cdb45c7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:23.579456 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.579425 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:23.579627 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.579579 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 17:08:23.579627 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.579599 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 17:08:23.579627 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.579609 2565 projected.go:196] Error preparing data for projected volume kube-api-access-5qqdt for pod openshift-network-diagnostics/network-check-target-26bm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:23.579757 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:23.579663 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt podName:decf50e7-9d50-402a-b7cd-e1c5589772b9 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:24.579650032 +0000 UTC m=+4.193194315 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqdt" (UniqueName: "kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt") pod "network-check-target-26bm7" (UID: "decf50e7-9d50-402a-b7cd-e1c5589772b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:23.844978 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.844950 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58c69f2f_02ff_4043_a868_aca57a713daa.slice/crio-695508acdcf1ebf935bde304ae829041bd0e9e8ddb36a9118cfee4f535cd3616 WatchSource:0}: Error finding container 695508acdcf1ebf935bde304ae829041bd0e9e8ddb36a9118cfee4f535cd3616: Status 404 returned error can't find the container with id 695508acdcf1ebf935bde304ae829041bd0e9e8ddb36a9118cfee4f535cd3616 Jul 21 17:08:23.846019 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.845985 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013607f3_00b2_48cb_9e89_d8a3cddafd22.slice/crio-35c360c85ceccef589d722643ee0268226c2b913309bdfa68fd8de8f8df6aed9 WatchSource:0}: Error finding container 35c360c85ceccef589d722643ee0268226c2b913309bdfa68fd8de8f8df6aed9: Status 404 returned error can't find the container with id 35c360c85ceccef589d722643ee0268226c2b913309bdfa68fd8de8f8df6aed9 Jul 21 17:08:23.846649 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.846618 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod163f8c01_98b8_4a45_b3b5_461b057c4349.slice/crio-83630b55ba97e9a12b2a4116cd860057b7027c4a625e762234a65a5b2aea3dcd WatchSource:0}: Error finding container 83630b55ba97e9a12b2a4116cd860057b7027c4a625e762234a65a5b2aea3dcd: Status 404 returned error can't find the container with id 83630b55ba97e9a12b2a4116cd860057b7027c4a625e762234a65a5b2aea3dcd Jul 21 17:08:23.877250 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.877200 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93b92f78_4af1_41c9_9464_fa3b32effaa6.slice/crio-e4f4262bab855a86023e4d05da84a45f6ee3c9b84437e4c31c1514e5f6a6857a WatchSource:0}: Error finding container e4f4262bab855a86023e4d05da84a45f6ee3c9b84437e4c31c1514e5f6a6857a: Status 404 returned error can't find the container with id e4f4262bab855a86023e4d05da84a45f6ee3c9b84437e4c31c1514e5f6a6857a Jul 21 17:08:23.877956 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.877930 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b818316_6b2d_4e37_b96e_ff47692a0fd9.slice/crio-2e97af31fc1d1f88dbc87c4d0703bfe6417ab13c128877e73815929011652080 WatchSource:0}: Error finding container 2e97af31fc1d1f88dbc87c4d0703bfe6417ab13c128877e73815929011652080: Status 404 returned error can't find the container with id 2e97af31fc1d1f88dbc87c4d0703bfe6417ab13c128877e73815929011652080 Jul 21 17:08:23.878839 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.878816 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49b95fd_0f47_423f_a36a_536f9ef79bf9.slice/crio-1b365c092fdc9f8ddc6cc068e725b435d09f5dd75771cd3d2be045c3351fe97b WatchSource:0}: Error finding container 1b365c092fdc9f8ddc6cc068e725b435d09f5dd75771cd3d2be045c3351fe97b: Status 404 returned error can't find the container with id 1b365c092fdc9f8ddc6cc068e725b435d09f5dd75771cd3d2be045c3351fe97b Jul 21 17:08:23.880341 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.880272 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47e09fe0_e9a5_43d9_98db_4a6d131eff09.slice/crio-35a5c2ae1c519fa2593f87acad7694aff7ca816cc6717f7e6267227db29010ad WatchSource:0}: Error finding container 35a5c2ae1c519fa2593f87acad7694aff7ca816cc6717f7e6267227db29010ad: Status 404 returned error can't find the container with id 35a5c2ae1c519fa2593f87acad7694aff7ca816cc6717f7e6267227db29010ad Jul 21 17:08:23.881093 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.881059 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d06660_fbc8_4804_96be_69563b12276d.slice/crio-999f8a99231597ddab0959338337f0d2a9fd51a8b7ed674bdc1339f1472e1753 WatchSource:0}: Error finding container 999f8a99231597ddab0959338337f0d2a9fd51a8b7ed674bdc1339f1472e1753: Status 404 returned error can't find the container with id 999f8a99231597ddab0959338337f0d2a9fd51a8b7ed674bdc1339f1472e1753 Jul 21 17:08:23.883648 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:23.883623 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd62be561_533b_4be0_8ab5_6ea408150db5.slice/crio-cb7eb99d3433e74798e4b22eee96a94082e09d700f6e05a120af53d841c56566 WatchSource:0}: Error finding container cb7eb99d3433e74798e4b22eee96a94082e09d700f6e05a120af53d841c56566: Status 404 returned error can't find the container with id cb7eb99d3433e74798e4b22eee96a94082e09d700f6e05a120af53d841c56566 Jul 21 17:08:23.914211 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.914165 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:03:22 +0000 UTC" deadline="2028-04-04 13:08:55.039614346 +0000 UTC" Jul 21 17:08:23.914211 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:23.914207 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14948h0m31.125409973s" Jul 21 17:08:24.027259 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.027033 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:24.027414 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.027072 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:24.027414 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.027371 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:24.027507 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.027416 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:24.036519 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.036371 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" event={"ID":"70278ed5d98e32abf177b8a325b5bf17","Type":"ContainerStarted","Data":"245cad7e7c92a73b91df9c60b22eb089ee41c69c64d4933e0de240f29a81f5de"} Jul 21 17:08:24.037780 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.037754 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wjfzj" event={"ID":"b4d06660-fbc8-4804-96be-69563b12276d","Type":"ContainerStarted","Data":"999f8a99231597ddab0959338337f0d2a9fd51a8b7ed674bdc1339f1472e1753"} Jul 21 17:08:24.038865 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.038834 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" event={"ID":"47e09fe0-e9a5-43d9-98db-4a6d131eff09","Type":"ContainerStarted","Data":"35a5c2ae1c519fa2593f87acad7694aff7ca816cc6717f7e6267227db29010ad"} Jul 21 17:08:24.039987 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.039953 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wqfb5" event={"ID":"d49b95fd-0f47-423f-a36a-536f9ef79bf9","Type":"ContainerStarted","Data":"1b365c092fdc9f8ddc6cc068e725b435d09f5dd75771cd3d2be045c3351fe97b"} Jul 21 17:08:24.041140 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.041117 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r99rt" event={"ID":"93b92f78-4af1-41c9-9464-fa3b32effaa6","Type":"ContainerStarted","Data":"e4f4262bab855a86023e4d05da84a45f6ee3c9b84437e4c31c1514e5f6a6857a"} Jul 21 17:08:24.042364 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.042332 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerStarted","Data":"cb7eb99d3433e74798e4b22eee96a94082e09d700f6e05a120af53d841c56566"} Jul 21 17:08:24.043433 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.043403 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7f57l" event={"ID":"8b818316-6b2d-4e37-b96e-ff47692a0fd9","Type":"ContainerStarted","Data":"2e97af31fc1d1f88dbc87c4d0703bfe6417ab13c128877e73815929011652080"} Jul 21 17:08:24.044501 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.044474 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" event={"ID":"163f8c01-98b8-4a45-b3b5-461b057c4349","Type":"ContainerStarted","Data":"83630b55ba97e9a12b2a4116cd860057b7027c4a625e762234a65a5b2aea3dcd"} Jul 21 17:08:24.045497 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.045475 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5vmr7" event={"ID":"013607f3-00b2-48cb-9e89-d8a3cddafd22","Type":"ContainerStarted","Data":"35c360c85ceccef589d722643ee0268226c2b913309bdfa68fd8de8f8df6aed9"} Jul 21 17:08:24.046519 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.046501 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"695508acdcf1ebf935bde304ae829041bd0e9e8ddb36a9118cfee4f535cd3616"} Jul 21 17:08:24.053192 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.053137 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-109.ec2.internal" podStartSLOduration=2.053120437 podStartE2EDuration="2.053120437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:24.052939136 +0000 UTC m=+3.666483436" watchObservedRunningTime="2026-07-21 17:08:24.053120437 +0000 UTC m=+3.666664737" Jul 21 17:08:24.486008 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.485859 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:24.486008 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.485994 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:24.486819 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.486580 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:24.486819 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.486665 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret podName:ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:26.486645604 +0000 UTC m=+6.100189884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret") pod "global-pull-secret-syncer-slspc" (UID: "ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4") : object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:24.486819 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.486594 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:24.486819 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.486777 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs podName:17870260-a36e-451c-93ac-56c19cdb45c7 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:26.486723539 +0000 UTC m=+6.100267821 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs") pod "network-metrics-daemon-xrl6x" (UID: "17870260-a36e-451c-93ac-56c19cdb45c7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:24.587655 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.587610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:24.587860 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.587804 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 17:08:24.587860 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.587827 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 17:08:24.587860 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.587842 2565 projected.go:196] Error preparing data for projected volume kube-api-access-5qqdt for pod openshift-network-diagnostics/network-check-target-26bm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:24.588078 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:24.587908 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt podName:decf50e7-9d50-402a-b7cd-e1c5589772b9 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:26.587886338 +0000 UTC m=+6.201430660 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqdt" (UniqueName: "kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt") pod "network-check-target-26bm7" (UID: "decf50e7-9d50-402a-b7cd-e1c5589772b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:24.915420 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.915324 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-20 17:03:22 +0000 UTC" deadline="2028-02-26 08:31:52.732352442 +0000 UTC" Jul 21 17:08:24.915420 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:24.915388 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14031h23m27.816968345s" Jul 21 17:08:25.029718 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:25.029552 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:25.029718 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:25.029683 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:25.072449 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:25.072305 2565 generic.go:356] "Generic (PLEG): container finished" podID="9ee6ac37bca8309a5190f95f0c1bc299" containerID="4464c108c0fe56225999cf10281697aaca5bd79dc8f25c439991163a0c5cc2b3" exitCode=0 Jul 21 17:08:25.072449 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:25.072380 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" event={"ID":"9ee6ac37bca8309a5190f95f0c1bc299","Type":"ContainerDied","Data":"4464c108c0fe56225999cf10281697aaca5bd79dc8f25c439991163a0c5cc2b3"} Jul 21 17:08:26.027412 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:26.026701 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:26.027412 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.026855 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:26.027412 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:26.026701 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:26.027412 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.027363 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:26.088725 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:26.088673 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" event={"ID":"9ee6ac37bca8309a5190f95f0c1bc299","Type":"ContainerStarted","Data":"d786756b72d0808d8d153822c1a2cd883af1f6bd50c6503b928b54c9dcca2087"} Jul 21 17:08:26.509632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:26.509532 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:26.509632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:26.509595 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:26.509865 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.509725 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:26.509865 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.509790 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret podName:ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:30.509771439 +0000 UTC m=+10.123315739 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret") pod "global-pull-secret-syncer-slspc" (UID: "ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4") : object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:26.509972 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.509865 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:26.509972 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.509897 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs podName:17870260-a36e-451c-93ac-56c19cdb45c7 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:30.509886486 +0000 UTC m=+10.123430765 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs") pod "network-metrics-daemon-xrl6x" (UID: "17870260-a36e-451c-93ac-56c19cdb45c7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:26.611499 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:26.610807 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:26.611499 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.611024 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 17:08:26.611499 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.611043 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 17:08:26.611499 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.611057 2565 projected.go:196] Error preparing data for projected volume kube-api-access-5qqdt for pod openshift-network-diagnostics/network-check-target-26bm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:26.611499 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:26.611118 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt podName:decf50e7-9d50-402a-b7cd-e1c5589772b9 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:30.611099945 +0000 UTC m=+10.224644238 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqdt" (UniqueName: "kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt") pod "network-check-target-26bm7" (UID: "decf50e7-9d50-402a-b7cd-e1c5589772b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:27.029181 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:27.029147 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:27.029674 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:27.029298 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:28.026518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:28.026484 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:28.026693 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:28.026625 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:28.026693 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:28.026671 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:28.026937 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:28.026883 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:29.027212 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:29.027177 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:29.027692 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:29.027341 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:30.027179 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:30.026687 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:30.027179 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.026820 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:30.027179 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:30.026687 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:30.027738 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.027194 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:30.545781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:30.545737 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:30.545982 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:30.545806 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:30.545982 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.545954 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:30.546098 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.546015 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret podName:ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:38.545997503 +0000 UTC m=+18.159541789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret") pod "global-pull-secret-syncer-slspc" (UID: "ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4") : object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:30.546381 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.546360 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:30.546461 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.546420 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs podName:17870260-a36e-451c-93ac-56c19cdb45c7 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:38.546404245 +0000 UTC m=+18.159948523 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs") pod "network-metrics-daemon-xrl6x" (UID: "17870260-a36e-451c-93ac-56c19cdb45c7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:30.646923 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:30.646882 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:30.647163 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.647133 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 17:08:30.647163 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.647161 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 17:08:30.647364 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.647174 2565 projected.go:196] Error preparing data for projected volume kube-api-access-5qqdt for pod openshift-network-diagnostics/network-check-target-26bm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:30.647364 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:30.647249 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt podName:decf50e7-9d50-402a-b7cd-e1c5589772b9 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:38.647216737 +0000 UTC m=+18.260761033 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqdt" (UniqueName: "kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt") pod "network-check-target-26bm7" (UID: "decf50e7-9d50-402a-b7cd-e1c5589772b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:31.027935 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:31.027838 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:31.028411 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:31.027959 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:32.026741 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:32.026703 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:32.026904 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:32.026746 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:32.026904 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:32.026863 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:32.027012 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:32.026983 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:33.026798 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:33.026287 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:33.026798 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:33.026416 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:34.026874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:34.026834 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:34.027416 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:34.026955 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:34.027416 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:34.027010 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:34.027416 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:34.027125 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:35.027184 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:35.027148 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:35.027660 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:35.027299 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:36.026766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:36.026727 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:36.026952 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:36.026778 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:36.026952 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:36.026871 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:36.027080 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:36.026990 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:37.026911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:37.026872 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:37.027373 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:37.027009 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:38.026706 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:38.026666 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:38.026706 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:38.026710 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:38.026967 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.026798 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:38.026967 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.026951 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:38.610050 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:38.610011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:38.610231 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:38.610060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:38.610231 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.610192 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:38.610231 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.610212 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:38.610373 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.610279 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs podName:17870260-a36e-451c-93ac-56c19cdb45c7 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:54.610258192 +0000 UTC m=+34.223802476 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs") pod "network-metrics-daemon-xrl6x" (UID: "17870260-a36e-451c-93ac-56c19cdb45c7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:38.610373 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.610301 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret podName:ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:54.610290592 +0000 UTC m=+34.223834871 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret") pod "global-pull-secret-syncer-slspc" (UID: "ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4") : object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:38.710608 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:38.710566 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:38.710809 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.710751 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 17:08:38.710809 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.710774 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 17:08:38.710809 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.710784 2565 projected.go:196] Error preparing data for projected volume kube-api-access-5qqdt for pod openshift-network-diagnostics/network-check-target-26bm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:38.711023 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:38.710847 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt podName:decf50e7-9d50-402a-b7cd-e1c5589772b9 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:54.710824938 +0000 UTC m=+34.324369238 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqdt" (UniqueName: "kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt") pod "network-check-target-26bm7" (UID: "decf50e7-9d50-402a-b7cd-e1c5589772b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:39.026842 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:39.026760 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:39.026982 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:39.026894 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:40.026431 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:40.026399 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:40.026641 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:40.026538 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:40.026641 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:40.026595 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:40.026750 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:40.026713 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:41.027781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:41.027741 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:41.028260 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:41.027863 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:42.027055 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:42.027013 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:42.027055 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:42.027035 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:42.027329 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:42.027130 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:42.027329 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:42.027306 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:43.026249 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:43.026203 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:43.026628 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:43.026332 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:44.026742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.026373 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:44.027444 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.026373 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:44.027444 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:44.026837 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:44.027444 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:44.026929 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:44.127454 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.127412 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wjfzj" event={"ID":"b4d06660-fbc8-4804-96be-69563b12276d","Type":"ContainerStarted","Data":"efa1fc16bdb8916072c03d1db46a81d9a95a25c72b4ec2a62c5df326e263b6cc"} Jul 21 17:08:44.129183 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.129146 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" event={"ID":"47e09fe0-e9a5-43d9-98db-4a6d131eff09","Type":"ContainerStarted","Data":"1b6511516d90c4ea7592c88c98df8a3455dfc9732f9b5b59d204702f61c9f221"} Jul 21 17:08:44.130863 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.130834 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-wqfb5" event={"ID":"d49b95fd-0f47-423f-a36a-536f9ef79bf9","Type":"ContainerStarted","Data":"be334d6d992453769c850770909f84d4ada815d104f08afc4faaf0eb081201fa"} Jul 21 17:08:44.132681 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.132651 2565 generic.go:356] "Generic (PLEG): container finished" podID="d62be561-533b-4be0-8ab5-6ea408150db5" containerID="460044a193e60db25412135ab1429a3ecaef38e473c42b228ddb202711bd8b48" exitCode=0 Jul 21 17:08:44.132803 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.132687 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerDied","Data":"460044a193e60db25412135ab1429a3ecaef38e473c42b228ddb202711bd8b48"} Jul 21 17:08:44.134716 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.134597 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7f57l" event={"ID":"8b818316-6b2d-4e37-b96e-ff47692a0fd9","Type":"ContainerStarted","Data":"fe35f4942afabd6960b778992ce01ce8ea3023d52d50cd3b9c4e515af4797612"} Jul 21 17:08:44.136368 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.136340 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" event={"ID":"163f8c01-98b8-4a45-b3b5-461b057c4349","Type":"ContainerStarted","Data":"9982dc3af7a46213c45a061a44d33dcc8762be9440a0cfd3862d65571b7483d3"} Jul 21 17:08:44.138289 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.138267 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5vmr7" event={"ID":"013607f3-00b2-48cb-9e89-d8a3cddafd22","Type":"ContainerStarted","Data":"9512f3b30e5860724f7125223cf1627f6aaf3e5949b4c0732e40ea1864a5b1ac"} Jul 21 17:08:44.142410 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.142336 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"4434c2004f285c61b7b11ef8f1490c7be9f803142ba009cb1f29f1f813e21ddc"} Jul 21 17:08:44.142529 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.142420 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"89edc357b3a38ad28cdc1c491879407f41e4fb7656709b623f60087cfa1ae0de"} Jul 21 17:08:44.142529 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.142473 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"12a6381e1234a9a8cc9dd6f2804ef109132e6933b34318e99670d3084741b3ab"} Jul 21 17:08:44.142529 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.142522 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"6492da9b654c43c9a5b0316d38576cc5628f90f067434ef855ae5fa1ce721cd0"} Jul 21 17:08:44.142682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.142571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"ff7167b4cabacb6215687f240aa4fa974fe6c680665bbfcba2722798bb74be7c"} Jul 21 17:08:44.142682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.142620 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"f40e5b8cbf6330630c32aa886c5ad64cfcfc214f40297311c74ae12b90f8ac98"} Jul 21 17:08:44.161451 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.161399 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-109.ec2.internal" podStartSLOduration=22.161381186 podStartE2EDuration="22.161381186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:26.106044456 +0000 UTC m=+5.719588757" watchObservedRunningTime="2026-07-21 17:08:44.161381186 +0000 UTC m=+23.774925486" Jul 21 17:08:44.161788 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.161769 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wjfzj" podStartSLOduration=3.951918907 podStartE2EDuration="23.161763994s" totalImagesPullingTime="19.209845087s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:44.161227647 +0000 UTC m=+23.774771948" watchObservedRunningTime="2026-07-21 17:08:44.161763994 +0000 UTC m=+23.775308295" Jul 21 17:08:44.203008 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.202900 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7f57l" podStartSLOduration=12.254979404 podStartE2EDuration="23.202883366s" totalImagesPullingTime="10.947903962s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:44.202089605 +0000 UTC m=+23.815633906" watchObservedRunningTime="2026-07-21 17:08:44.202883366 +0000 UTC m=+23.816427647" Jul 21 17:08:44.203557 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.203525 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-wqfb5" podStartSLOduration=4.02699237 podStartE2EDuration="23.203513865s" totalImagesPullingTime="19.176521495s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:44.182039645 +0000 UTC m=+23.795583946" watchObservedRunningTime="2026-07-21 17:08:44.203513865 +0000 UTC m=+23.817058165" Jul 21 17:08:44.222905 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.222851 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5vmr7" podStartSLOduration=4.038815357 podStartE2EDuration="23.222835559s" totalImagesPullingTime="19.184020202s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:44.222636252 +0000 UTC m=+23.836180554" watchObservedRunningTime="2026-07-21 17:08:44.222835559 +0000 UTC m=+23.836379856" Jul 21 17:08:44.283551 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.283494 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mkhk5" podStartSLOduration=4.106131702 podStartE2EDuration="23.283479438s" totalImagesPullingTime="19.177347736s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:44.251029791 +0000 UTC m=+23.864574094" watchObservedRunningTime="2026-07-21 17:08:44.283479438 +0000 UTC m=+23.897023741" Jul 21 17:08:44.589704 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.589672 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 21 17:08:44.941728 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.941611 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-21T17:08:44.589695967Z","UUID":"c3fb8cbd-6543-46bf-bed8-1345ca7f78d6","Handler":null,"Name":"","Endpoint":""} Jul 21 17:08:44.943365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.943342 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 21 17:08:44.943498 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:44.943380 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 21 17:08:45.027315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:45.027282 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:45.027825 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:45.027413 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:45.146374 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:45.146336 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r99rt" event={"ID":"93b92f78-4af1-41c9-9464-fa3b32effaa6","Type":"ContainerStarted","Data":"f01e6a6aaa79dbd67eebb304920c96894143b96bd4be441a9f8a7af72815da00"} Jul 21 17:08:45.148563 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:45.148526 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" event={"ID":"163f8c01-98b8-4a45-b3b5-461b057c4349","Type":"ContainerStarted","Data":"8c9a975735ac445a37778dd0ad0aaf08ffee23cb47d69dc071748d2a73128f58"} Jul 21 17:08:46.026434 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:46.026394 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:46.026777 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:46.026394 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:46.026777 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:46.026533 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:46.026777 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:46.026622 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:46.154123 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:46.154009 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" event={"ID":"163f8c01-98b8-4a45-b3b5-461b057c4349","Type":"ContainerStarted","Data":"691a1ff8a8272b4a4ef0b1d9c836f160f34b0fa48eb54f0b9dd0d1dfefb3825f"} Jul 21 17:08:46.158401 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:46.158363 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"3b3c04912cc636b654056baedfc7c54144b7c6e5ea8dab71839bf6dfdd52bef1"} Jul 21 17:08:46.180093 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:46.180038 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tlwfj" podStartSLOduration=3.8298285659999998 podStartE2EDuration="25.180021785s" totalImagesPullingTime="21.350193219s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:46.179726939 +0000 UTC m=+25.793271241" watchObservedRunningTime="2026-07-21 17:08:46.180021785 +0000 UTC m=+25.793566087" Jul 21 17:08:46.180532 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:46.180486 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r99rt" podStartSLOduration=6.003732834 podStartE2EDuration="25.180473313s" totalImagesPullingTime="19.176740479s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:45.178098475 +0000 UTC m=+24.791642776" watchObservedRunningTime="2026-07-21 17:08:46.180473313 +0000 UTC m=+25.794017613" Jul 21 17:08:47.026274 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:47.026221 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:47.026447 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:47.026354 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:47.728933 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:47.728893 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:47.729532 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:47.729503 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:48.027154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:48.026970 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:48.027338 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:48.027014 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:48.027338 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:48.027280 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:48.027440 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:48.027348 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:48.162661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:48.162627 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:48.163268 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:48.163228 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7f57l" Jul 21 17:08:49.026731 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:49.026695 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:49.027596 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:49.026838 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:49.166741 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:49.166647 2565 generic.go:356] "Generic (PLEG): container finished" podID="d62be561-533b-4be0-8ab5-6ea408150db5" containerID="42bf7cadbbdd92930e9d9dcc9a735891297ecf53f43a4260d9633c2e86a9cea7" exitCode=0 Jul 21 17:08:49.166741 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:49.166722 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerDied","Data":"42bf7cadbbdd92930e9d9dcc9a735891297ecf53f43a4260d9633c2e86a9cea7"} Jul 21 17:08:49.170703 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:49.170469 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" event={"ID":"58c69f2f-02ff-4043-a868-aca57a713daa","Type":"ContainerStarted","Data":"f618e08f24b30ecc9a36fa08d5819e51a28c6bc7c4f19a3ae28bf108675fac95"} Jul 21 17:08:49.225325 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:49.225268 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" podStartSLOduration=8.889163441 podStartE2EDuration="28.225232487s" totalImagesPullingTime="19.336069046s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:08:49.225183718 +0000 UTC m=+28.838728019" watchObservedRunningTime="2026-07-21 17:08:49.225232487 +0000 UTC m=+28.838776820" Jul 21 17:08:50.026740 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.026704 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:50.027257 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.026704 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:50.027257 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:50.026816 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:50.027257 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:50.026923 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:50.173038 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.173010 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:50.173193 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.173090 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:50.173193 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.173140 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:50.188366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.188338 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:50.189435 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.189414 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:08:50.882580 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.882548 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-26bm7"] Jul 21 17:08:50.882817 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.882724 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:50.882883 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:50.882837 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:50.885720 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.885694 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-slspc"] Jul 21 17:08:50.885946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.885806 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:50.885946 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:50.885900 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:50.887406 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.887379 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xrl6x"] Jul 21 17:08:50.887525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:50.887490 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:50.887595 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:50.887578 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:51.176493 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:51.176451 2565 generic.go:356] "Generic (PLEG): container finished" podID="d62be561-533b-4be0-8ab5-6ea408150db5" containerID="721436e5bfaf75606d05dccf49e90c3be33e3a405990d4a65c0461128d5a3557" exitCode=0 Jul 21 17:08:51.176916 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:51.176536 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerDied","Data":"721436e5bfaf75606d05dccf49e90c3be33e3a405990d4a65c0461128d5a3557"} Jul 21 17:08:52.026955 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:52.026918 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:52.026955 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:52.026945 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:52.027214 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:52.027051 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:52.027214 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:52.027170 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:53.026663 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:53.026422 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:53.027059 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:53.026763 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:53.184154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:53.184112 2565 generic.go:356] "Generic (PLEG): container finished" podID="d62be561-533b-4be0-8ab5-6ea408150db5" containerID="5de12f74456d84148c279a8f54af5dda67f7452955c921b8c44e06d118880231" exitCode=0 Jul 21 17:08:53.184333 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:53.184173 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerDied","Data":"5de12f74456d84148c279a8f54af5dda67f7452955c921b8c44e06d118880231"} Jul 21 17:08:54.026547 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:54.026511 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:54.026730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:54.026511 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:54.026730 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.026662 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xrl6x" podUID="17870260-a36e-451c-93ac-56c19cdb45c7" Jul 21 17:08:54.026730 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.026687 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-slspc" podUID="ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4" Jul 21 17:08:54.635173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:54.635128 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:54.635378 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:54.635202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:54.635378 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.635310 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:54.635378 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.635310 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:54.635378 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.635375 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret podName:ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:26.635353424 +0000 UTC m=+66.248897717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret") pod "global-pull-secret-syncer-slspc" (UID: "ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4") : object "kube-system"/"original-pull-secret" not registered Jul 21 17:08:54.635511 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.635398 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs podName:17870260-a36e-451c-93ac-56c19cdb45c7 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:26.635390421 +0000 UTC m=+66.248934700 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs") pod "network-metrics-daemon-xrl6x" (UID: "17870260-a36e-451c-93ac-56c19cdb45c7") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 21 17:08:54.736566 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:54.736534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:54.736732 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.736715 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 21 17:08:54.736777 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.736735 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 21 17:08:54.736777 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.736746 2565 projected.go:196] Error preparing data for projected volume kube-api-access-5qqdt for pod openshift-network-diagnostics/network-check-target-26bm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:54.736846 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:54.736801 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt podName:decf50e7-9d50-402a-b7cd-e1c5589772b9 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:26.736784203 +0000 UTC m=+66.350328502 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5qqdt" (UniqueName: "kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt") pod "network-check-target-26bm7" (UID: "decf50e7-9d50-402a-b7cd-e1c5589772b9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 21 17:08:55.026576 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.026539 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:55.026766 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:55.026677 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-26bm7" podUID="decf50e7-9d50-402a-b7cd-e1c5589772b9" Jul 21 17:08:55.695404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.695370 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-109.ec2.internal" event="NodeReady" Jul 21 17:08:55.695572 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.695530 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 21 17:08:55.756506 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.756466 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-849f95f869-c8vl4"] Jul 21 17:08:55.760277 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.760233 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.765597 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.765438 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 21 17:08:55.765597 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.765474 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 21 17:08:55.765597 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.765494 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-h5krw\"" Jul 21 17:08:55.766946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.766414 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 21 17:08:55.771758 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.771737 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 21 17:08:55.781288 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.781255 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fsxck"] Jul 21 17:08:55.785564 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.785537 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:55.786157 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.786094 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-849f95f869-c8vl4"] Jul 21 17:08:55.790256 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.790215 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 21 17:08:55.790562 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.790533 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 21 17:08:55.790661 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.790557 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jzcnj\"" Jul 21 17:08:55.790887 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.790871 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 21 17:08:55.798167 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.798143 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b2pfj"] Jul 21 17:08:55.805633 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.805606 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:55.810060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.810035 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 21 17:08:55.810405 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.810388 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bn6n8\"" Jul 21 17:08:55.811291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.811270 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 21 17:08:55.821500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.821468 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fsxck"] Jul 21 17:08:55.833208 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.833177 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2pfj"] Jul 21 17:08:55.845833 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.845746 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-image-registry-private-configuration\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.845833 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.845807 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-bound-sa-token\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.846062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.845833 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.846062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.845897 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-trusted-ca\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.846062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.845943 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-ca-trust-extracted\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.846062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.845960 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-certificates\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.846062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.845982 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-installation-pull-secrets\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.846062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.846009 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qws6v\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-kube-api-access-qws6v\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947179 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947143 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c57qd\" (UniqueName: \"kubernetes.io/projected/01e78e36-df87-42ea-91d9-d661ade09e86-kube-api-access-c57qd\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:55.947393 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947190 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c21775ac-3111-443d-a5cb-b9d7641e6e56-cert\") pod \"ingress-canary-fsxck\" (UID: \"c21775ac-3111-443d-a5cb-b9d7641e6e56\") " pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:55.947393 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947229 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e78e36-df87-42ea-91d9-d661ade09e86-tmp-dir\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:55.947393 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947270 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-trusted-ca\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947393 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947349 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-ca-trust-extracted\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947393 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-certificates\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947638 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-installation-pull-secrets\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947638 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947435 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qws6v\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-kube-api-access-qws6v\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947729 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947675 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8xhxt\" (UniqueName: \"kubernetes.io/projected/c21775ac-3111-443d-a5cb-b9d7641e6e56-kube-api-access-8xhxt\") pod \"ingress-canary-fsxck\" (UID: \"c21775ac-3111-443d-a5cb-b9d7641e6e56\") " pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:55.947729 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947716 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:55.947825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947748 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-image-registry-private-configuration\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-bound-sa-token\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947925 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947829 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/01e78e36-df87-42ea-91d9-d661ade09e86-config-volume\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:55.947925 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947854 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-ca-trust-extracted\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.947925 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.947860 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.948063 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:55.947988 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 17:08:55.948063 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:55.948003 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-849f95f869-c8vl4: secret "image-registry-tls" not found Jul 21 17:08:55.948160 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:55.948117 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls podName:a073ed17-ba21-4e1c-87f1-0ba13d0bfad1 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:56.448089523 +0000 UTC m=+36.061633815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls") pod "image-registry-849f95f869-c8vl4" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1") : secret "image-registry-tls" not found Jul 21 17:08:55.948362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.948340 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-trusted-ca\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.948527 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.948501 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-certificates\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.952128 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.952106 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-installation-pull-secrets\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.952267 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.952135 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-image-registry-private-configuration\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.964550 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.964517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qws6v\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-kube-api-access-qws6v\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:55.964686 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:55.964611 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-bound-sa-token\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:56.026711 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.026676 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:08:56.026711 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.026710 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:08:56.029613 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.029587 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 17:08:56.029741 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.029642 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wk959\"" Jul 21 17:08:56.029741 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.029695 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 17:08:56.048749 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.048719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e78e36-df87-42ea-91d9-d661ade09e86-tmp-dir\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.048909 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.048818 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8xhxt\" (UniqueName: \"kubernetes.io/projected/c21775ac-3111-443d-a5cb-b9d7641e6e56-kube-api-access-8xhxt\") pod \"ingress-canary-fsxck\" (UID: \"c21775ac-3111-443d-a5cb-b9d7641e6e56\") " pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:56.048909 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.048845 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.048909 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.048874 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/01e78e36-df87-42ea-91d9-d661ade09e86-config-volume\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.049049 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.048918 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c57qd\" (UniqueName: \"kubernetes.io/projected/01e78e36-df87-42ea-91d9-d661ade09e86-kube-api-access-c57qd\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.049049 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.048944 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c21775ac-3111-443d-a5cb-b9d7641e6e56-cert\") pod \"ingress-canary-fsxck\" (UID: \"c21775ac-3111-443d-a5cb-b9d7641e6e56\") " pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:56.049049 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:56.048986 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 17:08:56.049180 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:56.049057 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls podName:01e78e36-df87-42ea-91d9-d661ade09e86 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:56.5490382 +0000 UTC m=+36.162582493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls") pod "dns-default-b2pfj" (UID: "01e78e36-df87-42ea-91d9-d661ade09e86") : secret "dns-default-metrics-tls" not found Jul 21 17:08:56.049180 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.049080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/01e78e36-df87-42ea-91d9-d661ade09e86-tmp-dir\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.049592 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.049565 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/01e78e36-df87-42ea-91d9-d661ade09e86-config-volume\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.051872 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.051843 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c21775ac-3111-443d-a5cb-b9d7641e6e56-cert\") pod \"ingress-canary-fsxck\" (UID: \"c21775ac-3111-443d-a5cb-b9d7641e6e56\") " pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:56.058573 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.058515 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c57qd\" (UniqueName: \"kubernetes.io/projected/01e78e36-df87-42ea-91d9-d661ade09e86-kube-api-access-c57qd\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.058697 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.058648 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8xhxt\" (UniqueName: \"kubernetes.io/projected/c21775ac-3111-443d-a5cb-b9d7641e6e56-kube-api-access-8xhxt\") pod \"ingress-canary-fsxck\" (UID: \"c21775ac-3111-443d-a5cb-b9d7641e6e56\") " pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:56.097061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.096970 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fsxck" Jul 21 17:08:56.452717 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.452679 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:56.452860 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:56.452845 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 17:08:56.452908 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:56.452864 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-849f95f869-c8vl4: secret "image-registry-tls" not found Jul 21 17:08:56.452969 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:56.452931 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls podName:a073ed17-ba21-4e1c-87f1-0ba13d0bfad1 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:57.452906663 +0000 UTC m=+37.066450944 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls") pod "image-registry-849f95f869-c8vl4" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1") : secret "image-registry-tls" not found Jul 21 17:08:56.553766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:56.553731 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:56.553967 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:56.553902 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 17:08:56.554028 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:56.553969 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls podName:01e78e36-df87-42ea-91d9-d661ade09e86 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:57.553951583 +0000 UTC m=+37.167495865 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls") pod "dns-default-b2pfj" (UID: "01e78e36-df87-42ea-91d9-d661ade09e86") : secret "dns-default-metrics-tls" not found Jul 21 17:08:57.026647 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:57.026608 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:08:57.029856 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:57.029829 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9bczc\"" Jul 21 17:08:57.030470 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:57.030452 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 17:08:57.030739 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:57.030718 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 17:08:57.462430 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:57.462395 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:57.462556 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:57.462546 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 17:08:57.462588 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:57.462562 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-849f95f869-c8vl4: secret "image-registry-tls" not found Jul 21 17:08:57.462642 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:57.462620 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls podName:a073ed17-ba21-4e1c-87f1-0ba13d0bfad1 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:59.462601802 +0000 UTC m=+39.076146086 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls") pod "image-registry-849f95f869-c8vl4" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1") : secret "image-registry-tls" not found Jul 21 17:08:57.503461 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:57.503435 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fsxck"] Jul 21 17:08:57.507266 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:08:57.507223 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc21775ac_3111_443d_a5cb_b9d7641e6e56.slice/crio-225322acc3c0c04b0983e064d47d87152e4681f466c0de3b863e8173be272407 WatchSource:0}: Error finding container 225322acc3c0c04b0983e064d47d87152e4681f466c0de3b863e8173be272407: Status 404 returned error can't find the container with id 225322acc3c0c04b0983e064d47d87152e4681f466c0de3b863e8173be272407 Jul 21 17:08:57.563768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:57.563718 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:57.563940 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:57.563863 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 17:08:57.563940 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:57.563935 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls podName:01e78e36-df87-42ea-91d9-d661ade09e86 nodeName:}" failed. No retries permitted until 2026-07-21 17:08:59.563915741 +0000 UTC m=+39.177460021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls") pod "dns-default-b2pfj" (UID: "01e78e36-df87-42ea-91d9-d661ade09e86") : secret "dns-default-metrics-tls" not found Jul 21 17:08:58.198321 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:58.198282 2565 generic.go:356] "Generic (PLEG): container finished" podID="d62be561-533b-4be0-8ab5-6ea408150db5" containerID="b10672ddc38535ba9064173e5e9d808208fc5bc2552c2610af5b9f8f160fb8c9" exitCode=0 Jul 21 17:08:58.199032 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:58.198370 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerDied","Data":"b10672ddc38535ba9064173e5e9d808208fc5bc2552c2610af5b9f8f160fb8c9"} Jul 21 17:08:58.199767 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:58.199732 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fsxck" event={"ID":"c21775ac-3111-443d-a5cb-b9d7641e6e56","Type":"ContainerStarted","Data":"225322acc3c0c04b0983e064d47d87152e4681f466c0de3b863e8173be272407"} Jul 21 17:08:59.205731 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:59.205691 2565 generic.go:356] "Generic (PLEG): container finished" podID="d62be561-533b-4be0-8ab5-6ea408150db5" containerID="185391aa49105c55ac6c927de4778e1e1816e75f63a8069fa87c102901cc5d35" exitCode=0 Jul 21 17:08:59.206181 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:59.205781 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerDied","Data":"185391aa49105c55ac6c927de4778e1e1816e75f63a8069fa87c102901cc5d35"} Jul 21 17:08:59.479036 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:59.478951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:08:59.479202 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:59.479095 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 17:08:59.479202 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:59.479110 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-849f95f869-c8vl4: secret "image-registry-tls" not found Jul 21 17:08:59.479202 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:59.479165 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls podName:a073ed17-ba21-4e1c-87f1-0ba13d0bfad1 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:03.479148783 +0000 UTC m=+43.092693065 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls") pod "image-registry-849f95f869-c8vl4" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1") : secret "image-registry-tls" not found Jul 21 17:08:59.580056 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:08:59.580020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:08:59.580271 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:59.580200 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 17:08:59.580341 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:08:59.580296 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls podName:01e78e36-df87-42ea-91d9-d661ade09e86 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:03.580279094 +0000 UTC m=+43.193823391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls") pod "dns-default-b2pfj" (UID: "01e78e36-df87-42ea-91d9-d661ade09e86") : secret "dns-default-metrics-tls" not found Jul 21 17:09:00.211922 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:00.211890 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xhljt" event={"ID":"d62be561-533b-4be0-8ab5-6ea408150db5","Type":"ContainerStarted","Data":"f6c08e675974e10e0ee091439515f4c1d4defedfb637416fcbb00d4ef0aaf017"} Jul 21 17:09:00.241882 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:00.241816 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xhljt" podStartSLOduration=8.589424187 podStartE2EDuration="39.241794698s" totalImagesPullingTime="30.652370511s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:00.24118099 +0000 UTC m=+39.854725293" watchObservedRunningTime="2026-07-21 17:09:00.241794698 +0000 UTC m=+39.855339001" Jul 21 17:09:01.216094 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:01.215912 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fsxck" event={"ID":"c21775ac-3111-443d-a5cb-b9d7641e6e56","Type":"ContainerStarted","Data":"e564ac166aaf91ed6462236d6e0e3a269af935e828078d2430fdf951e53c58c1"} Jul 21 17:09:01.238763 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:01.238709 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fsxck" podStartSLOduration=3.593220799 podStartE2EDuration="6.238692715s" totalImagesPullingTime="2.645471916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:01.238117149 +0000 UTC m=+40.851661444" watchObservedRunningTime="2026-07-21 17:09:01.238692715 +0000 UTC m=+40.852237016" Jul 21 17:09:03.146957 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.146921 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m4qjh"] Jul 21 17:09:03.179485 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.179450 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m4qjh"] Jul 21 17:09:03.179647 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.179583 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.182471 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.182450 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 21 17:09:03.185000 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.184980 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 21 17:09:03.185571 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.185554 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 21 17:09:03.185689 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.185594 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t495x\"" Jul 21 17:09:03.185764 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.185596 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 21 17:09:03.307356 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.307317 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3e8a413-7456-422e-a38b-f29321c499a0-crio-socket\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.307537 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.307368 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.307537 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.307409 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jrhd\" (UniqueName: \"kubernetes.io/projected/b3e8a413-7456-422e-a38b-f29321c499a0-kube-api-access-7jrhd\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.307537 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.307433 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3e8a413-7456-422e-a38b-f29321c499a0-data-volume\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.307537 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.307451 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.407916 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.407823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.407916 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.407907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3e8a413-7456-422e-a38b-f29321c499a0-crio-socket\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.407938 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.407950 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.407959 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jrhd\" (UniqueName: \"kubernetes.io/projected/b3e8a413-7456-422e-a38b-f29321c499a0-kube-api-access-7jrhd\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.408035 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:03.908014209 +0000 UTC m=+43.521558505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : configmap "kube-rbac-proxy" not found Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.408062 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.408085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3e8a413-7456-422e-a38b-f29321c499a0-data-volume\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.408109 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:03.908096595 +0000 UTC m=+43.521640874 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : secret "insights-runtime-extractor-tls" not found Jul 21 17:09:03.408150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.408111 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b3e8a413-7456-422e-a38b-f29321c499a0-crio-socket\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.408571 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.408430 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3e8a413-7456-422e-a38b-f29321c499a0-data-volume\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.446196 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.446165 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jrhd\" (UniqueName: \"kubernetes.io/projected/b3e8a413-7456-422e-a38b-f29321c499a0-kube-api-access-7jrhd\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.509434 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.509394 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:09:03.509578 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.509547 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 17:09:03.509578 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.509566 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-849f95f869-c8vl4: secret "image-registry-tls" not found Jul 21 17:09:03.509661 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.509623 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls podName:a073ed17-ba21-4e1c-87f1-0ba13d0bfad1 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:11.509606514 +0000 UTC m=+51.123150793 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls") pod "image-registry-849f95f869-c8vl4" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1") : secret "image-registry-tls" not found Jul 21 17:09:03.610254 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.610213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:09:03.610440 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.610391 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 17:09:03.610506 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.610470 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls podName:01e78e36-df87-42ea-91d9-d661ade09e86 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:11.610449178 +0000 UTC m=+51.223993473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls") pod "dns-default-b2pfj" (UID: "01e78e36-df87-42ea-91d9-d661ade09e86") : secret "dns-default-metrics-tls" not found Jul 21 17:09:03.912322 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.912288 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.912506 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:03.912351 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:03.912506 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.912453 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 17:09:03.912506 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.912472 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 17:09:03.912654 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.912520 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:04.912503643 +0000 UTC m=+44.526047922 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : secret "insights-runtime-extractor-tls" not found Jul 21 17:09:03.912654 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:03.912535 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:04.912528333 +0000 UTC m=+44.526072611 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : configmap "kube-rbac-proxy" not found Jul 21 17:09:04.920232 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:04.920189 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:04.920232 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:04.920264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:04.920773 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:04.920380 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 17:09:04.920773 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:04.920393 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 17:09:04.920773 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:04.920453 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:06.920438139 +0000 UTC m=+46.533982419 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : configmap "kube-rbac-proxy" not found Jul 21 17:09:04.920773 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:04.920468 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:06.920461915 +0000 UTC m=+46.534006194 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : secret "insights-runtime-extractor-tls" not found Jul 21 17:09:05.667956 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:05.667928 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5vmr7_013607f3-00b2-48cb-9e89-d8a3cddafd22/dns-node-resolver/0.log" Jul 21 17:09:06.936220 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:06.936185 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:06.936919 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:06.936254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:06.936919 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:06.936358 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 17:09:06.936919 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:06.936368 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 17:09:06.936919 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:06.936431 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:10.93641168 +0000 UTC m=+50.549955961 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : secret "insights-runtime-extractor-tls" not found Jul 21 17:09:06.936919 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:06.936453 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:10.936443057 +0000 UTC m=+50.549987337 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : configmap "kube-rbac-proxy" not found Jul 21 17:09:07.054689 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:07.054667 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wqfb5_d49b95fd-0f47-423f-a36a-536f9ef79bf9/node-ca/0.log" Jul 21 17:09:07.852416 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:07.852392 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fsxck_c21775ac-3111-443d-a5cb-b9d7641e6e56/serve-healthcheck-canary/0.log" Jul 21 17:09:10.972458 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:10.972417 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:10.972839 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:10.972469 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:10.972839 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:10.972585 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 17:09:10.972839 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:10.972612 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 21 17:09:10.972839 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:10.972666 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:18.972647692 +0000 UTC m=+58.586191974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : secret "insights-runtime-extractor-tls" not found Jul 21 17:09:10.972839 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:10.972690 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:18.97267577 +0000 UTC m=+58.586220049 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : configmap "kube-rbac-proxy" not found Jul 21 17:09:11.577964 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:11.577910 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") pod \"image-registry-849f95f869-c8vl4\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:09:11.578163 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:11.578068 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 21 17:09:11.578163 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:11.578086 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-849f95f869-c8vl4: secret "image-registry-tls" not found Jul 21 17:09:11.578163 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:11.578161 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls podName:a073ed17-ba21-4e1c-87f1-0ba13d0bfad1 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:27.578144689 +0000 UTC m=+67.191688967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls") pod "image-registry-849f95f869-c8vl4" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1") : secret "image-registry-tls" not found Jul 21 17:09:11.678965 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:11.678927 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:09:11.679134 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:11.679084 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 21 17:09:11.679173 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:11.679156 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls podName:01e78e36-df87-42ea-91d9-d661ade09e86 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:27.679140981 +0000 UTC m=+67.292685260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls") pod "dns-default-b2pfj" (UID: "01e78e36-df87-42ea-91d9-d661ade09e86") : secret "dns-default-metrics-tls" not found Jul 21 17:09:19.036295 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:19.036264 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:19.036736 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:19.036314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:19.036736 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:19.036425 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 21 17:09:19.036736 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:19.036488 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls podName:b3e8a413-7456-422e-a38b-f29321c499a0 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:35.036471449 +0000 UTC m=+74.650015728 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m4qjh" (UID: "b3e8a413-7456-422e-a38b-f29321c499a0") : secret "insights-runtime-extractor-tls" not found Jul 21 17:09:19.036888 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:19.036870 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b3e8a413-7456-422e-a38b-f29321c499a0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:22.198679 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:22.198648 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-q45lm" Jul 21 17:09:25.585176 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.585140 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn"] Jul 21 17:09:25.588122 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.588100 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.590558 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.590537 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn"] Jul 21 17:09:25.592608 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.592593 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.595200 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.595177 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 21 17:09:25.595385 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.595364 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 21 17:09:25.596302 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.596279 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-q8wcx\"" Jul 21 17:09:25.596302 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.596292 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 21 17:09:25.596430 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.596308 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 21 17:09:25.598531 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.598495 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 21 17:09:25.598531 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.598528 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 21 17:09:25.598709 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.598643 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 21 17:09:25.598982 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.598781 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 21 17:09:25.609160 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.609139 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn"] Jul 21 17:09:25.627438 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.627411 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn"] Jul 21 17:09:25.688614 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688578 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbxpr\" (UniqueName: \"kubernetes.io/projected/4a819c3a-18bc-4e18-a887-ce006a58d371-kube-api-access-rbxpr\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.688812 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688636 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb004bfa-7776-4f97-a977-68dcfdece45d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-854d657c8d-m8csn\" (UID: \"cb004bfa-7776-4f97-a977-68dcfdece45d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.688812 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688660 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-ca\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.688915 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688818 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a819c3a-18bc-4e18-a887-ce006a58d371-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.688915 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.688915 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688882 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7d42\" (UniqueName: \"kubernetes.io/projected/cb004bfa-7776-4f97-a977-68dcfdece45d-kube-api-access-s7d42\") pod \"managed-serviceaccount-addon-agent-854d657c8d-m8csn\" (UID: \"cb004bfa-7776-4f97-a977-68dcfdece45d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.689031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688930 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-hub\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.689031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.688961 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.764306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.764265 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-849f95f869-c8vl4"] Jul 21 17:09:25.764518 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:25.764496 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-849f95f869-c8vl4" podUID="a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" Jul 21 17:09:25.789846 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.789812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbxpr\" (UniqueName: \"kubernetes.io/projected/4a819c3a-18bc-4e18-a887-ce006a58d371-kube-api-access-rbxpr\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.789932 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.789863 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb004bfa-7776-4f97-a977-68dcfdece45d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-854d657c8d-m8csn\" (UID: \"cb004bfa-7776-4f97-a977-68dcfdece45d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.789932 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.789880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-ca\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.790031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.789953 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a819c3a-18bc-4e18-a887-ce006a58d371-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.790031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.789983 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.790031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.790008 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7d42\" (UniqueName: \"kubernetes.io/projected/cb004bfa-7776-4f97-a977-68dcfdece45d-kube-api-access-s7d42\") pod \"managed-serviceaccount-addon-agent-854d657c8d-m8csn\" (UID: \"cb004bfa-7776-4f97-a977-68dcfdece45d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.790031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.790025 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-hub\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.790207 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.790051 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.790816 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.790781 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/4a819c3a-18bc-4e18-a887-ce006a58d371-additional-service-ca\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.792723 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.792694 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-ca\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.792813 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.792745 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.792853 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.792833 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cb004bfa-7776-4f97-a977-68dcfdece45d-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-854d657c8d-m8csn\" (UID: \"cb004bfa-7776-4f97-a977-68dcfdece45d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.793031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.793011 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.793086 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.793072 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/4a819c3a-18bc-4e18-a887-ce006a58d371-hub\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.811713 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.811677 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbxpr\" (UniqueName: \"kubernetes.io/projected/4a819c3a-18bc-4e18-a887-ce006a58d371-kube-api-access-rbxpr\") pod \"cluster-proxy-proxy-agent-5d6bcdb887-2bsmn\" (UID: \"4a819c3a-18bc-4e18-a887-ce006a58d371\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:25.811841 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.811783 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7d42\" (UniqueName: \"kubernetes.io/projected/cb004bfa-7776-4f97-a977-68dcfdece45d-kube-api-access-s7d42\") pod \"managed-serviceaccount-addon-agent-854d657c8d-m8csn\" (UID: \"cb004bfa-7776-4f97-a977-68dcfdece45d\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.904866 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.904779 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" Jul 21 17:09:25.910272 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:25.910250 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" Jul 21 17:09:26.035969 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.035936 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn"] Jul 21 17:09:26.039045 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:26.039019 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb004bfa_7776_4f97_a977_68dcfdece45d.slice/crio-3c8f6183961c8fc151c3454eb294e2a1d2f16f65daf547a269c2b118069d7a96 WatchSource:0}: Error finding container 3c8f6183961c8fc151c3454eb294e2a1d2f16f65daf547a269c2b118069d7a96: Status 404 returned error can't find the container with id 3c8f6183961c8fc151c3454eb294e2a1d2f16f65daf547a269c2b118069d7a96 Jul 21 17:09:26.054360 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.054330 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn"] Jul 21 17:09:26.057115 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:26.057088 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4a819c3a_18bc_4e18_a887_ce006a58d371.slice/crio-1c013e6c467082eff46af2b1f96f6bb030424c9a12381ed2b70da8aba6394bd6 WatchSource:0}: Error finding container 1c013e6c467082eff46af2b1f96f6bb030424c9a12381ed2b70da8aba6394bd6: Status 404 returned error can't find the container with id 1c013e6c467082eff46af2b1f96f6bb030424c9a12381ed2b70da8aba6394bd6 Jul 21 17:09:26.274355 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.274318 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" event={"ID":"4a819c3a-18bc-4e18-a887-ce006a58d371","Type":"ContainerStarted","Data":"1c013e6c467082eff46af2b1f96f6bb030424c9a12381ed2b70da8aba6394bd6"} Jul 21 17:09:26.275477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.275457 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" event={"ID":"cb004bfa-7776-4f97-a977-68dcfdece45d","Type":"ContainerStarted","Data":"3c8f6183961c8fc151c3454eb294e2a1d2f16f65daf547a269c2b118069d7a96"} Jul 21 17:09:26.275541 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.275478 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:09:26.282497 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.282477 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:09:26.395119 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395087 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-bound-sa-token\") pod \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " Jul 21 17:09:26.395357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395130 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-trusted-ca\") pod \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " Jul 21 17:09:26.395357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395154 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-installation-pull-secrets\") pod \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " Jul 21 17:09:26.395357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395178 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-ca-trust-extracted\") pod \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " Jul 21 17:09:26.395357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395306 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-certificates\") pod \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " Jul 21 17:09:26.395584 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395384 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qws6v\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-kube-api-access-qws6v\") pod \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " Jul 21 17:09:26.395584 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395416 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-image-registry-private-configuration\") pod \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\" (UID: \"a073ed17-ba21-4e1c-87f1-0ba13d0bfad1\") " Jul 21 17:09:26.395584 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395503 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:09:26.395751 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395732 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-ca-trust-extracted\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:26.395811 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395729 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:09:26.396001 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.395962 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:09:26.397501 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.397472 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:09:26.397589 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.397529 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:09:26.397589 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.397531 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:09:26.397589 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.397549 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-kube-api-access-qws6v" (OuterVolumeSpecName: "kube-api-access-qws6v") pod "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" (UID: "a073ed17-ba21-4e1c-87f1-0ba13d0bfad1"). InnerVolumeSpecName "kube-api-access-qws6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:09:26.496499 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.496464 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-trusted-ca\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:26.496499 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.496494 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-installation-pull-secrets\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:26.496499 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.496503 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-certificates\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:26.496723 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.496512 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qws6v\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-kube-api-access-qws6v\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:26.496723 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.496522 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-image-registry-private-configuration\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:26.496723 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.496531 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-bound-sa-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:26.698559 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.698473 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:09:26.698559 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.698522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:09:26.701061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.701035 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 21 17:09:26.701184 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.701059 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 21 17:09:26.711461 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.711436 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4-original-pull-secret\") pod \"global-pull-secret-syncer-slspc\" (UID: \"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4\") " pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:09:26.711567 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.711443 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/17870260-a36e-451c-93ac-56c19cdb45c7-metrics-certs\") pod \"network-metrics-daemon-xrl6x\" (UID: \"17870260-a36e-451c-93ac-56c19cdb45c7\") " pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:09:26.799164 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.799122 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:09:26.802195 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.802171 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 21 17:09:26.812233 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.812209 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 21 17:09:26.822056 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.822025 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qqdt\" (UniqueName: \"kubernetes.io/projected/decf50e7-9d50-402a-b7cd-e1c5589772b9-kube-api-access-5qqdt\") pod \"network-check-target-26bm7\" (UID: \"decf50e7-9d50-402a-b7cd-e1c5589772b9\") " pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:09:26.942138 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.942102 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wk959\"" Jul 21 17:09:26.945443 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.945424 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-slspc" Jul 21 17:09:26.949949 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:26.949931 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xrl6x" Jul 21 17:09:27.040306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.040280 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-9bczc\"" Jul 21 17:09:27.047584 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.047559 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:09:27.083717 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.083678 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xrl6x"] Jul 21 17:09:27.087079 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:27.087046 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17870260_a36e_451c_93ac_56c19cdb45c7.slice/crio-f3665121acf816391490f0d3326e7894e5841540ddfde3a783d6f02c2b72c33f WatchSource:0}: Error finding container f3665121acf816391490f0d3326e7894e5841540ddfde3a783d6f02c2b72c33f: Status 404 returned error can't find the container with id f3665121acf816391490f0d3326e7894e5841540ddfde3a783d6f02c2b72c33f Jul 21 17:09:27.100819 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.100792 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-slspc"] Jul 21 17:09:27.181231 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.181202 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-26bm7"] Jul 21 17:09:27.183815 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:27.183781 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddecf50e7_9d50_402a_b7cd_e1c5589772b9.slice/crio-549e3a6e05d654366c79cfb64271f76498ebf3bb1b4a6c64b08805605dc6b938 WatchSource:0}: Error finding container 549e3a6e05d654366c79cfb64271f76498ebf3bb1b4a6c64b08805605dc6b938: Status 404 returned error can't find the container with id 549e3a6e05d654366c79cfb64271f76498ebf3bb1b4a6c64b08805605dc6b938 Jul 21 17:09:27.278845 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.278809 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-slspc" event={"ID":"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4","Type":"ContainerStarted","Data":"0b737ce0974878e63a8b148a481e7a27ce74939eea4f084ddc6a0f5cbbc2673f"} Jul 21 17:09:27.279996 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.279962 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrl6x" event={"ID":"17870260-a36e-451c-93ac-56c19cdb45c7","Type":"ContainerStarted","Data":"f3665121acf816391490f0d3326e7894e5841540ddfde3a783d6f02c2b72c33f"} Jul 21 17:09:27.281077 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.281055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-26bm7" event={"ID":"decf50e7-9d50-402a-b7cd-e1c5589772b9","Type":"ContainerStarted","Data":"549e3a6e05d654366c79cfb64271f76498ebf3bb1b4a6c64b08805605dc6b938"} Jul 21 17:09:27.281077 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.281075 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-849f95f869-c8vl4" Jul 21 17:09:27.313766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.313736 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-849f95f869-c8vl4"] Jul 21 17:09:27.319436 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.319411 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-849f95f869-c8vl4"] Jul 21 17:09:27.408163 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.408131 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1-registry-tls\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:09:27.709958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.709867 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:09:27.715135 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.715069 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/01e78e36-df87-42ea-91d9-d661ade09e86-metrics-tls\") pod \"dns-default-b2pfj\" (UID: \"01e78e36-df87-42ea-91d9-d661ade09e86\") " pod="openshift-dns/dns-default-b2pfj" Jul 21 17:09:27.919575 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.919343 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-bn6n8\"" Jul 21 17:09:27.927280 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:27.926958 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b2pfj" Jul 21 17:09:28.469282 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:28.469174 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b2pfj"] Jul 21 17:09:28.479344 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:28.479273 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e78e36_df87_42ea_91d9_d661ade09e86.slice/crio-753619bba59baeec7c57a6731da12a531d13eddbe9f78a151cdb278ce54cd739 WatchSource:0}: Error finding container 753619bba59baeec7c57a6731da12a531d13eddbe9f78a151cdb278ce54cd739: Status 404 returned error can't find the container with id 753619bba59baeec7c57a6731da12a531d13eddbe9f78a151cdb278ce54cd739 Jul 21 17:09:29.034699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:29.034663 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a073ed17-ba21-4e1c-87f1-0ba13d0bfad1" path="/var/lib/kubelet/pods/a073ed17-ba21-4e1c-87f1-0ba13d0bfad1/volumes" Jul 21 17:09:29.292345 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:29.292169 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2pfj" event={"ID":"01e78e36-df87-42ea-91d9-d661ade09e86","Type":"ContainerStarted","Data":"753619bba59baeec7c57a6731da12a531d13eddbe9f78a151cdb278ce54cd739"} Jul 21 17:09:29.295485 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:29.295327 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrl6x" event={"ID":"17870260-a36e-451c-93ac-56c19cdb45c7","Type":"ContainerStarted","Data":"578d4bc029085e4533c0e876489f2af5f4b3a1691dadb6b3d4731208bf784f3a"} Jul 21 17:09:29.295485 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:29.295389 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xrl6x" event={"ID":"17870260-a36e-451c-93ac-56c19cdb45c7","Type":"ContainerStarted","Data":"e9a76078eee889ce8a2ff156d8bc38a1e3cf12ce80c9ca6213688b74ecc88f8f"} Jul 21 17:09:29.320641 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:29.320557 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xrl6x" podStartSLOduration=67.064558441 podStartE2EDuration="1m8.320535054s" totalImagesPullingTime="1.255976613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:29.319288572 +0000 UTC m=+68.932832867" watchObservedRunningTime="2026-07-21 17:09:29.320535054 +0000 UTC m=+68.934079350" Jul 21 17:09:35.074305 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.074256 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:35.076537 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.076515 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b3e8a413-7456-422e-a38b-f29321c499a0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m4qjh\" (UID: \"b3e8a413-7456-422e-a38b-f29321c499a0\") " pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:35.292559 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.292533 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t495x\"" Jul 21 17:09:35.299736 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.299708 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m4qjh" Jul 21 17:09:35.319726 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.319692 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" event={"ID":"4a819c3a-18bc-4e18-a887-ce006a58d371","Type":"ContainerStarted","Data":"2a92d9ee413acb19c97c842d412adaffec03f6596b05caecd0f1e2778d1d9a9d"} Jul 21 17:09:35.319888 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.319739 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" event={"ID":"4a819c3a-18bc-4e18-a887-ce006a58d371","Type":"ContainerStarted","Data":"084ce230111dca8f56d6ebe266a62a2452507702778e26c3cd345cf0b542de53"} Jul 21 17:09:35.319888 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.319757 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" event={"ID":"4a819c3a-18bc-4e18-a887-ce006a58d371","Type":"ContainerStarted","Data":"24afe7e31025b543a745ae3ad6d5208ac7a88a70f8921cda5e000d7977c987b5"} Jul 21 17:09:35.321326 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.321279 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-slspc" event={"ID":"ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4","Type":"ContainerStarted","Data":"d582970f379144264ff1989be60b2865edf42cfb60176e2133348e9c668def8e"} Jul 21 17:09:35.323029 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.323001 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2pfj" event={"ID":"01e78e36-df87-42ea-91d9-d661ade09e86","Type":"ContainerStarted","Data":"bb31185a4c7d7eefaf3406f19616f456b1cc7945149e5eae310b5fa00453e58e"} Jul 21 17:09:35.323155 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.323055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b2pfj" event={"ID":"01e78e36-df87-42ea-91d9-d661ade09e86","Type":"ContainerStarted","Data":"56b2543c6fabd3c639d0312328a76052f3e5d4a30ee4df06ec937bef6de6f52a"} Jul 21 17:09:35.323155 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.323088 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b2pfj" Jul 21 17:09:35.324428 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.324407 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-26bm7" event={"ID":"decf50e7-9d50-402a-b7cd-e1c5589772b9","Type":"ContainerStarted","Data":"3d5f87080713d3a7b4553b95fa38c688e8a727daea1c303602b6e5a172559ca3"} Jul 21 17:09:35.324561 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.324500 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:09:35.325892 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.325868 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" event={"ID":"cb004bfa-7776-4f97-a977-68dcfdece45d","Type":"ContainerStarted","Data":"97a9942bf70677d7176105df6ab8e8d0f68d7ca61e814ba673606e027516f64e"} Jul 21 17:09:35.341089 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.341040 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-5d6bcdb887-2bsmn" podStartSLOduration=1.719555951 podStartE2EDuration="10.341025266s" totalImagesPullingTime="8.621469315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:35.33989332 +0000 UTC m=+74.953437633" watchObservedRunningTime="2026-07-21 17:09:35.341025266 +0000 UTC m=+74.954569545" Jul 21 17:09:35.356544 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.356498 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-26bm7" podStartSLOduration=66.861536004 podStartE2EDuration="1m14.356479687s" totalImagesPullingTime="7.494943683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:35.355880555 +0000 UTC m=+74.969424855" watchObservedRunningTime="2026-07-21 17:09:35.356479687 +0000 UTC m=+74.970023988" Jul 21 17:09:35.372304 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.372200 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-854d657c8d-m8csn" podStartSLOduration=1.7331978289999999 podStartE2EDuration="10.372182265s" totalImagesPullingTime="8.638984436s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:09:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:35.370991024 +0000 UTC m=+74.984535326" watchObservedRunningTime="2026-07-21 17:09:35.372182265 +0000 UTC m=+74.985726565" Jul 21 17:09:35.391305 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.391227 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b2pfj" podStartSLOduration=34.14099211 podStartE2EDuration="40.391209972s" totalImagesPullingTime="6.250217862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:08:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:35.390598058 +0000 UTC m=+75.004142358" watchObservedRunningTime="2026-07-21 17:09:35.391209972 +0000 UTC m=+75.004754273" Jul 21 17:09:35.412327 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.412276 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-slspc" podStartSLOduration=66.838461546 podStartE2EDuration="1m14.412261997s" totalImagesPullingTime="7.573800451s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:08:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:35.411860776 +0000 UTC m=+75.025405080" watchObservedRunningTime="2026-07-21 17:09:35.412261997 +0000 UTC m=+75.025806299" Jul 21 17:09:35.433498 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:35.433463 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m4qjh"] Jul 21 17:09:35.436413 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:35.436385 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3e8a413_7456_422e_a38b_f29321c499a0.slice/crio-f93f805f9ffc0040f0c0bd203cd8d05c740b7100a700bc45b8ffa27905e8c240 WatchSource:0}: Error finding container f93f805f9ffc0040f0c0bd203cd8d05c740b7100a700bc45b8ffa27905e8c240: Status 404 returned error can't find the container with id f93f805f9ffc0040f0c0bd203cd8d05c740b7100a700bc45b8ffa27905e8c240 Jul 21 17:09:36.213097 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.213051 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2"] Jul 21 17:09:36.216345 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.216317 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.220134 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.220113 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 21 17:09:36.220482 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.220460 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-tsn54\"" Jul 21 17:09:36.225263 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.224979 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 21 17:09:36.226268 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.226129 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 21 17:09:36.226383 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.226349 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 21 17:09:36.226854 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.226689 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 21 17:09:36.234962 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.234868 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2"] Jul 21 17:09:36.282720 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.282683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj4zd\" (UniqueName: \"kubernetes.io/projected/177c485f-7bfa-464f-b941-254d609f4121-kube-api-access-fj4zd\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.282720 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.282740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.282966 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.282805 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.282966 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.282824 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/177c485f-7bfa-464f-b941-254d609f4121-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.331716 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.331678 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m4qjh" event={"ID":"b3e8a413-7456-422e-a38b-f29321c499a0","Type":"ContainerStarted","Data":"f93f805f9ffc0040f0c0bd203cd8d05c740b7100a700bc45b8ffa27905e8c240"} Jul 21 17:09:36.383823 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.383583 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.383823 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.383629 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/177c485f-7bfa-464f-b941-254d609f4121-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.383823 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.383762 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj4zd\" (UniqueName: \"kubernetes.io/projected/177c485f-7bfa-464f-b941-254d609f4121-kube-api-access-fj4zd\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.384117 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.383881 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.384117 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:36.383981 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 21 17:09:36.384117 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:36.384044 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls podName:177c485f-7bfa-464f-b941-254d609f4121 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:36.884022385 +0000 UTC m=+76.497566664 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-rz5z2" (UID: "177c485f-7bfa-464f-b941-254d609f4121") : secret "prometheus-operator-tls" not found Jul 21 17:09:36.384591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.384544 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/177c485f-7bfa-464f-b941-254d609f4121-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.387328 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.387294 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.395143 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.395112 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj4zd\" (UniqueName: \"kubernetes.io/projected/177c485f-7bfa-464f-b941-254d609f4121-kube-api-access-fj4zd\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.889103 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:36.889058 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:36.889327 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:36.889221 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 21 17:09:36.889327 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:36.889324 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls podName:177c485f-7bfa-464f-b941-254d609f4121 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:37.889307426 +0000 UTC m=+77.502851705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-rz5z2" (UID: "177c485f-7bfa-464f-b941-254d609f4121") : secret "prometheus-operator-tls" not found Jul 21 17:09:37.895693 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:37.895651 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:37.896218 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:37.895791 2565 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 21 17:09:37.896218 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:37.895877 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls podName:177c485f-7bfa-464f-b941-254d609f4121 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:39.895856975 +0000 UTC m=+79.509401258 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-rz5z2" (UID: "177c485f-7bfa-464f-b941-254d609f4121") : secret "prometheus-operator-tls" not found Jul 21 17:09:39.913941 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:39.913896 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:39.916810 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:39.916779 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/177c485f-7bfa-464f-b941-254d609f4121-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-rz5z2\" (UID: \"177c485f-7bfa-464f-b941-254d609f4121\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:40.139056 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:40.139014 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" Jul 21 17:09:40.405080 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:40.405055 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2"] Jul 21 17:09:40.408597 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:40.408566 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod177c485f_7bfa_464f_b941_254d609f4121.slice/crio-103c89ff6d81ad9708a7789f37b3938ef9327ac6a98d8c3dac73bc8bfcceb2a2 WatchSource:0}: Error finding container 103c89ff6d81ad9708a7789f37b3938ef9327ac6a98d8c3dac73bc8bfcceb2a2: Status 404 returned error can't find the container with id 103c89ff6d81ad9708a7789f37b3938ef9327ac6a98d8c3dac73bc8bfcceb2a2 Jul 21 17:09:41.352134 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:41.352093 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" event={"ID":"177c485f-7bfa-464f-b941-254d609f4121","Type":"ContainerStarted","Data":"103c89ff6d81ad9708a7789f37b3938ef9327ac6a98d8c3dac73bc8bfcceb2a2"} Jul 21 17:09:41.354575 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:41.354502 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m4qjh" event={"ID":"b3e8a413-7456-422e-a38b-f29321c499a0","Type":"ContainerStarted","Data":"4e1543dff32a846af2fa8683509209c5c4fa01b01375589828297ae5848c72ba"} Jul 21 17:09:42.360213 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:42.360178 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" event={"ID":"177c485f-7bfa-464f-b941-254d609f4121","Type":"ContainerStarted","Data":"f917d348e5ff2c85884ea382e701f57cc5dfef5701b9785cdd25042b7a48854f"} Jul 21 17:09:42.360670 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:42.360257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" event={"ID":"177c485f-7bfa-464f-b941-254d609f4121","Type":"ContainerStarted","Data":"ac76161646f9f2a492964b777411d600ec2104adc5685291e67ff935f5efc9f9"} Jul 21 17:09:42.379850 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:42.379797 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-rz5z2" podStartSLOduration=5.025348175 podStartE2EDuration="6.379781208s" totalImagesPullingTime="1.354433033s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:42.378881054 +0000 UTC m=+81.992425368" watchObservedRunningTime="2026-07-21 17:09:42.379781208 +0000 UTC m=+81.993325486" Jul 21 17:09:44.369377 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.369347 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m4qjh" event={"ID":"b3e8a413-7456-422e-a38b-f29321c499a0","Type":"ContainerStarted","Data":"4160e562ceab8f92d458910a585e3e26e234f9bb60df6b1be5392f0d1026928c"} Jul 21 17:09:44.716151 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.716068 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fzzj8"] Jul 21 17:09:44.719430 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.719407 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.724946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.724822 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 21 17:09:44.724946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.724841 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-rx78m\"" Jul 21 17:09:44.724946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.724916 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 21 17:09:44.725154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.724823 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 21 17:09:44.854010 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.853951 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-root\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854158 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854033 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854158 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854062 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-sys\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854158 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854086 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-wtmp\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854158 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854144 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-textfile\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854340 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854186 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh68d\" (UniqueName: \"kubernetes.io/projected/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-kube-api-access-lh68d\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854340 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854231 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-metrics-client-ca\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854340 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854327 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.854451 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.854362 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955295 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955253 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-metrics-client-ca\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955455 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955412 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955535 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955454 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955535 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955494 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-root\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955535 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955529 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955680 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:44.955556 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 17:09:44.955680 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955616 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-root\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955680 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:44.955623 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls podName:743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:45.455600604 +0000 UTC m=+85.069144898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls") pod "node-exporter-fzzj8" (UID: "743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1") : secret "node-exporter-tls" not found Jul 21 17:09:44.955680 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-sys\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955883 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955685 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-wtmp\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955883 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955723 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-textfile\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.955883 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh68d\" (UniqueName: \"kubernetes.io/projected/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-kube-api-access-lh68d\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.956046 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955870 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-wtmp\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.956046 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.955918 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-metrics-client-ca\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.956046 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.956007 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-sys\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.956193 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.956066 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-textfile\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.956193 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.956069 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-accelerators-collector-config\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.958046 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.958026 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:44.967113 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:44.967057 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh68d\" (UniqueName: \"kubernetes.io/projected/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-kube-api-access-lh68d\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:45.335549 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:45.335517 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b2pfj" Jul 21 17:09:45.460564 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:45.460526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:45.461030 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:45.460696 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 21 17:09:45.461030 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:45.460777 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls podName:743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1 nodeName:}" failed. No retries permitted until 2026-07-21 17:09:46.460756388 +0000 UTC m=+86.074300684 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls") pod "node-exporter-fzzj8" (UID: "743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1") : secret "node-exporter-tls" not found Jul 21 17:09:46.471345 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:46.471308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:46.474042 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:46.474015 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1-node-exporter-tls\") pod \"node-exporter-fzzj8\" (UID: \"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1\") " pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:46.530118 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:46.530081 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fzzj8" Jul 21 17:09:47.553303 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:47.553275 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod743c28b5_25dc_4a56_bc2a_9cd7f5c57cb1.slice/crio-10e43597f29daa6a22525f4634d97117d7289ca7e376ddd97e7b78073ed82bb9 WatchSource:0}: Error finding container 10e43597f29daa6a22525f4634d97117d7289ca7e376ddd97e7b78073ed82bb9: Status 404 returned error can't find the container with id 10e43597f29daa6a22525f4634d97117d7289ca7e376ddd97e7b78073ed82bb9 Jul 21 17:09:48.390533 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:48.390494 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m4qjh" event={"ID":"b3e8a413-7456-422e-a38b-f29321c499a0","Type":"ContainerStarted","Data":"6f23a70051b426d552418cb926216183e48ac80c7538c300dc4c903566660317"} Jul 21 17:09:48.391812 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:48.391755 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzzj8" event={"ID":"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1","Type":"ContainerStarted","Data":"10e43597f29daa6a22525f4634d97117d7289ca7e376ddd97e7b78073ed82bb9"} Jul 21 17:09:48.411294 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:48.411220 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m4qjh" podStartSLOduration=33.375427346 podStartE2EDuration="45.411203717s" totalImagesPullingTime="12.035776371s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:09:03 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:48.411002069 +0000 UTC m=+88.024546372" watchObservedRunningTime="2026-07-21 17:09:48.411203717 +0000 UTC m=+88.024748028" Jul 21 17:09:49.373482 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.373449 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp"] Jul 21 17:09:49.376448 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.376429 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:49.378533 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.378512 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 21 17:09:49.378632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.378568 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-gg7h5\"" Jul 21 17:09:49.386369 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.386349 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp"] Jul 21 17:09:49.396734 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.396710 2565 generic.go:356] "Generic (PLEG): container finished" podID="743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1" containerID="b2cb238123a3010c199a5576595a1ee510b07f56535c0c3b699cd834478c04f3" exitCode=0 Jul 21 17:09:49.396818 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.396790 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzzj8" event={"ID":"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1","Type":"ContainerDied","Data":"b2cb238123a3010c199a5576595a1ee510b07f56535c0c3b699cd834478c04f3"} Jul 21 17:09:49.493315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.493292 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx2gn\" (UniqueName: \"kubernetes.io/projected/05af373c-9955-4714-9604-4e7903dd202f-kube-api-access-lx2gn\") pod \"monitoring-plugin-7b7b7bc8bf-k9qgp\" (UID: \"05af373c-9955-4714-9604-4e7903dd202f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:49.493413 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.493340 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05af373c-9955-4714-9604-4e7903dd202f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k9qgp\" (UID: \"05af373c-9955-4714-9604-4e7903dd202f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:49.594603 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.594568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx2gn\" (UniqueName: \"kubernetes.io/projected/05af373c-9955-4714-9604-4e7903dd202f-kube-api-access-lx2gn\") pod \"monitoring-plugin-7b7b7bc8bf-k9qgp\" (UID: \"05af373c-9955-4714-9604-4e7903dd202f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:49.594603 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.594610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05af373c-9955-4714-9604-4e7903dd202f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k9qgp\" (UID: \"05af373c-9955-4714-9604-4e7903dd202f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:49.594850 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:49.594722 2565 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 21 17:09:49.594850 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:09:49.594783 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/05af373c-9955-4714-9604-4e7903dd202f-monitoring-plugin-cert podName:05af373c-9955-4714-9604-4e7903dd202f nodeName:}" failed. No retries permitted until 2026-07-21 17:09:50.094765619 +0000 UTC m=+89.708309898 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/05af373c-9955-4714-9604-4e7903dd202f-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-k9qgp" (UID: "05af373c-9955-4714-9604-4e7903dd202f") : secret "monitoring-plugin-cert" not found Jul 21 17:09:49.611253 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:49.611220 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx2gn\" (UniqueName: \"kubernetes.io/projected/05af373c-9955-4714-9604-4e7903dd202f-kube-api-access-lx2gn\") pod \"monitoring-plugin-7b7b7bc8bf-k9qgp\" (UID: \"05af373c-9955-4714-9604-4e7903dd202f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:50.097417 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:50.097373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05af373c-9955-4714-9604-4e7903dd202f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k9qgp\" (UID: \"05af373c-9955-4714-9604-4e7903dd202f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:50.099804 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:50.099786 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/05af373c-9955-4714-9604-4e7903dd202f-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-k9qgp\" (UID: \"05af373c-9955-4714-9604-4e7903dd202f\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:50.286864 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:50.286831 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:50.402281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:50.402251 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzzj8" event={"ID":"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1","Type":"ContainerStarted","Data":"3e10447440724109a4f4732209fa3c1b5106011fbaaee63e0e888e22905cbe61"} Jul 21 17:09:50.402719 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:50.402299 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fzzj8" event={"ID":"743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1","Type":"ContainerStarted","Data":"fbae52aa51cf8652ebfbbf21bb060b06bc514ec98863e71724f094c1b86a0517"} Jul 21 17:09:50.405835 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:50.405808 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp"] Jul 21 17:09:50.409414 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:09:50.409391 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05af373c_9955_4714_9604_4e7903dd202f.slice/crio-7f779307b1b60fd5167070967a37755b7c55d0913b01912d824622f64fa9d0f8 WatchSource:0}: Error finding container 7f779307b1b60fd5167070967a37755b7c55d0913b01912d824622f64fa9d0f8: Status 404 returned error can't find the container with id 7f779307b1b60fd5167070967a37755b7c55d0913b01912d824622f64fa9d0f8 Jul 21 17:09:50.422891 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:50.422854 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fzzj8" podStartSLOduration=5.42524202 podStartE2EDuration="6.422840265s" totalImagesPullingTime="936.96685ms" totalInitContainerRuntime="60.631395ms" isStatefulPod=true podCreationTimestamp="2026-07-21 17:09:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:50.421609901 +0000 UTC m=+90.035154203" watchObservedRunningTime="2026-07-21 17:09:50.422840265 +0000 UTC m=+90.036384566" Jul 21 17:09:51.407591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:51.407551 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" event={"ID":"05af373c-9955-4714-9604-4e7903dd202f","Type":"ContainerStarted","Data":"7f779307b1b60fd5167070967a37755b7c55d0913b01912d824622f64fa9d0f8"} Jul 21 17:09:52.411679 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:52.411647 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" event={"ID":"05af373c-9955-4714-9604-4e7903dd202f","Type":"ContainerStarted","Data":"07aa22772f07a218928688b551a996c18f98179e488fe0ebdadd2157e96c7359"} Jul 21 17:09:52.412075 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:52.411867 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:52.416463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:52.416441 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" Jul 21 17:09:52.429493 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:09:52.429446 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-k9qgp" podStartSLOduration=2.072035306 podStartE2EDuration="3.429434254s" totalImagesPullingTime="1.357398948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:09:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:09:52.429052438 +0000 UTC m=+92.042596738" watchObservedRunningTime="2026-07-21 17:09:52.429434254 +0000 UTC m=+92.042978555" Jul 21 17:10:06.334393 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:10:06.334358 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-26bm7" Jul 21 17:13:20.875909 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:13:20.875880 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 21 17:15:44.740294 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.740194 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2"] Jul 21 17:15:44.743160 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.743142 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:44.745659 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.745630 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 21 17:15:44.745766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.745689 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-wtbmj\"" Jul 21 17:15:44.745766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.745630 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 21 17:15:44.751409 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.751388 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2"] Jul 21 17:15:44.803469 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.803437 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjcrf\" (UniqueName: \"kubernetes.io/projected/2e83b1d5-8b44-4c76-8f8b-f51f08afa171-kube-api-access-gjcrf\") pod \"cert-manager-cainjector-559cdd8c87-d6wt2\" (UID: \"2e83b1d5-8b44-4c76-8f8b-f51f08afa171\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:44.803659 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.803492 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e83b1d5-8b44-4c76-8f8b-f51f08afa171-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d6wt2\" (UID: \"2e83b1d5-8b44-4c76-8f8b-f51f08afa171\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:44.904748 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.904707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e83b1d5-8b44-4c76-8f8b-f51f08afa171-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d6wt2\" (UID: \"2e83b1d5-8b44-4c76-8f8b-f51f08afa171\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:44.904937 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.904803 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gjcrf\" (UniqueName: \"kubernetes.io/projected/2e83b1d5-8b44-4c76-8f8b-f51f08afa171-kube-api-access-gjcrf\") pod \"cert-manager-cainjector-559cdd8c87-d6wt2\" (UID: \"2e83b1d5-8b44-4c76-8f8b-f51f08afa171\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:44.913635 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.913600 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2e83b1d5-8b44-4c76-8f8b-f51f08afa171-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-d6wt2\" (UID: \"2e83b1d5-8b44-4c76-8f8b-f51f08afa171\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:44.913772 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:44.913665 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjcrf\" (UniqueName: \"kubernetes.io/projected/2e83b1d5-8b44-4c76-8f8b-f51f08afa171-kube-api-access-gjcrf\") pod \"cert-manager-cainjector-559cdd8c87-d6wt2\" (UID: \"2e83b1d5-8b44-4c76-8f8b-f51f08afa171\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:45.052388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:45.052298 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" Jul 21 17:15:45.181736 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:45.181705 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2"] Jul 21 17:15:45.185070 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:15:45.185043 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2e83b1d5_8b44_4c76_8f8b_f51f08afa171.slice/crio-45cf5b6e6b2cfa10ef6a281f2b2330de4235593a49ecd8b2c32902aec1ab4af7 WatchSource:0}: Error finding container 45cf5b6e6b2cfa10ef6a281f2b2330de4235593a49ecd8b2c32902aec1ab4af7: Status 404 returned error can't find the container with id 45cf5b6e6b2cfa10ef6a281f2b2330de4235593a49ecd8b2c32902aec1ab4af7 Jul 21 17:15:45.187279 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:45.187255 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 17:15:45.529591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:45.529558 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" event={"ID":"2e83b1d5-8b44-4c76-8f8b-f51f08afa171","Type":"ContainerStarted","Data":"45cf5b6e6b2cfa10ef6a281f2b2330de4235593a49ecd8b2c32902aec1ab4af7"} Jul 21 17:15:48.543045 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:48.543007 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" event={"ID":"2e83b1d5-8b44-4c76-8f8b-f51f08afa171","Type":"ContainerStarted","Data":"ba1002152081f4c52cf5ef372a58a2e420ced998fe052135bfd8d3c297405efa"} Jul 21 17:15:48.560461 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:15:48.560397 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-d6wt2" podStartSLOduration=1.41464056 podStartE2EDuration="4.56034926s" totalImagesPullingTime="3.1457087s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:15:48.560150897 +0000 UTC m=+448.173695200" watchObservedRunningTime="2026-07-21 17:15:48.56034926 +0000 UTC m=+448.173893559" Jul 21 17:16:15.616029 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.615994 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt"] Jul 21 17:16:15.619063 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.619040 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.623880 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.623846 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 21 17:16:15.623880 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.623852 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 21 17:16:15.623880 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.623852 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-d7tnw\"" Jul 21 17:16:15.631536 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.631513 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 21 17:16:15.631731 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.631623 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 21 17:16:15.631807 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.631789 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 21 17:16:15.675394 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.675358 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt"] Jul 21 17:16:15.738504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.738463 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-metrics-cert\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.738504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.738508 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l4jb\" (UniqueName: \"kubernetes.io/projected/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-kube-api-access-7l4jb\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.738716 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.738532 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-cert\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.738716 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.738570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-manager-config\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.839020 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.838989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-manager-config\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.839213 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.839044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-metrics-cert\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.839213 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.839158 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l4jb\" (UniqueName: \"kubernetes.io/projected/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-kube-api-access-7l4jb\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.839213 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.839206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-cert\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.839777 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.839752 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-manager-config\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.841601 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.841580 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-metrics-cert\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.841700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.841651 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-cert\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.852335 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.852308 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l4jb\" (UniqueName: \"kubernetes.io/projected/ccf27e6b-dc1e-4da1-a6bf-383d43b02328-kube-api-access-7l4jb\") pod \"lws-controller-manager-646df98c5d-p9wtt\" (UID: \"ccf27e6b-dc1e-4da1-a6bf-383d43b02328\") " pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:15.927800 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:15.927710 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:16.068655 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.068622 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt"] Jul 21 17:16:16.072794 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:16:16.072763 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccf27e6b_dc1e_4da1_a6bf_383d43b02328.slice/crio-af55e59a21bff5daf9251a4dc5936bf15814d0337f32ded26156cce15fa5a3c4 WatchSource:0}: Error finding container af55e59a21bff5daf9251a4dc5936bf15814d0337f32ded26156cce15fa5a3c4: Status 404 returned error can't find the container with id af55e59a21bff5daf9251a4dc5936bf15814d0337f32ded26156cce15fa5a3c4 Jul 21 17:16:16.636978 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.636944 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" event={"ID":"ccf27e6b-dc1e-4da1-a6bf-383d43b02328","Type":"ContainerStarted","Data":"af55e59a21bff5daf9251a4dc5936bf15814d0337f32ded26156cce15fa5a3c4"} Jul 21 17:16:16.965711 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.965617 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp"] Jul 21 17:16:16.969093 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.969042 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:16.973934 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.973730 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 21 17:16:16.973934 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.973828 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-m8pfx\"" Jul 21 17:16:16.974381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.974363 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 21 17:16:16.974739 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.974590 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 21 17:16:16.974739 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:16.974723 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 21 17:16:17.049504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.049197 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/010c7fb0-4f11-4158-a9df-70ccf23fc509-webhook-cert\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.049504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.049311 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010c7fb0-4f11-4158-a9df-70ccf23fc509-tmp\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.049504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.049344 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/010c7fb0-4f11-4158-a9df-70ccf23fc509-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.049504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.049369 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkbhm\" (UniqueName: \"kubernetes.io/projected/010c7fb0-4f11-4158-a9df-70ccf23fc509-kube-api-access-wkbhm\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.049504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.049402 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/010c7fb0-4f11-4158-a9df-70ccf23fc509-manifests\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.057685 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.057655 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp"] Jul 21 17:16:17.150150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.150111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/010c7fb0-4f11-4158-a9df-70ccf23fc509-manifests\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.150367 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.150176 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/010c7fb0-4f11-4158-a9df-70ccf23fc509-webhook-cert\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.150367 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.150282 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010c7fb0-4f11-4158-a9df-70ccf23fc509-tmp\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.150367 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.150312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/010c7fb0-4f11-4158-a9df-70ccf23fc509-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.150367 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.150337 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkbhm\" (UniqueName: \"kubernetes.io/projected/010c7fb0-4f11-4158-a9df-70ccf23fc509-kube-api-access-wkbhm\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.150636 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.150610 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/010c7fb0-4f11-4158-a9df-70ccf23fc509-manifests\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.153258 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.153217 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/010c7fb0-4f11-4158-a9df-70ccf23fc509-webhook-cert\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.153878 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.153853 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/010c7fb0-4f11-4158-a9df-70ccf23fc509-tmp\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.153990 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.153910 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/010c7fb0-4f11-4158-a9df-70ccf23fc509-apiservice-cert\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.170867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.170834 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkbhm\" (UniqueName: \"kubernetes.io/projected/010c7fb0-4f11-4158-a9df-70ccf23fc509-kube-api-access-wkbhm\") pod \"opendatahub-operator-controller-manager-65b498688-wgbwp\" (UID: \"010c7fb0-4f11-4158-a9df-70ccf23fc509\") " pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.284063 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.284026 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:17.447097 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:17.447061 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp"] Jul 21 17:16:18.414260 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:16:18.414204 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010c7fb0_4f11_4158_a9df_70ccf23fc509.slice/crio-8b05f275205c0029421d8128969876987263dd762650985d8dff99c207b67853 WatchSource:0}: Error finding container 8b05f275205c0029421d8128969876987263dd762650985d8dff99c207b67853: Status 404 returned error can't find the container with id 8b05f275205c0029421d8128969876987263dd762650985d8dff99c207b67853 Jul 21 17:16:18.646167 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:18.646121 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" event={"ID":"010c7fb0-4f11-4158-a9df-70ccf23fc509","Type":"ContainerStarted","Data":"8b05f275205c0029421d8128969876987263dd762650985d8dff99c207b67853"} Jul 21 17:16:18.647610 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:18.647584 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" event={"ID":"ccf27e6b-dc1e-4da1-a6bf-383d43b02328","Type":"ContainerStarted","Data":"701c9df927c92a943a872d7bea28b71428983e144c70d70a7c72b71accf43038"} Jul 21 17:16:18.647730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:18.647646 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:18.669515 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:18.669422 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" podStartSLOduration=1.2696586619999999 podStartE2EDuration="3.669406386s" totalImagesPullingTime="2.399747724s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:16:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:16:18.668284628 +0000 UTC m=+478.281828932" watchObservedRunningTime="2026-07-21 17:16:18.669406386 +0000 UTC m=+478.282950687" Jul 21 17:16:21.660197 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:21.660109 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" event={"ID":"010c7fb0-4f11-4158-a9df-70ccf23fc509","Type":"ContainerStarted","Data":"0e807a21db56e1f2401f458002001f86f7b0f4b932b382a65729d6a6135e7c2c"} Jul 21 17:16:22.665543 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:22.665505 2565 generic.go:356] "Generic (PLEG): container finished" podID="010c7fb0-4f11-4158-a9df-70ccf23fc509" containerID="0e807a21db56e1f2401f458002001f86f7b0f4b932b382a65729d6a6135e7c2c" exitCode=0 Jul 21 17:16:22.665936 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:22.665597 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" event={"ID":"010c7fb0-4f11-4158-a9df-70ccf23fc509","Type":"ContainerDied","Data":"0e807a21db56e1f2401f458002001f86f7b0f4b932b382a65729d6a6135e7c2c"} Jul 21 17:16:23.673992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:23.673956 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" event={"ID":"010c7fb0-4f11-4158-a9df-70ccf23fc509","Type":"ContainerStarted","Data":"c4c1c50d4e166868260b1495f09bb066b6ce0b8b91647584ccd03c58aaa7904d"} Jul 21 17:16:23.674514 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:23.674071 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:23.696703 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:23.696653 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" podStartSLOduration=3.387372081 podStartE2EDuration="7.696635124s" totalImagesPullingTime="3.267968991s" totalInitContainerRuntime="1.041294052s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:16:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:16:23.694675979 +0000 UTC m=+483.308220280" watchObservedRunningTime="2026-07-21 17:16:23.696635124 +0000 UTC m=+483.310179424" Jul 21 17:16:29.655320 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:29.655287 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-646df98c5d-p9wtt" Jul 21 17:16:34.681691 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:34.681662 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-65b498688-wgbwp" Jul 21 17:16:40.625962 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.625921 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fpb9n"] Jul 21 17:16:40.643599 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.643567 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:40.644804 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.644755 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fpb9n"] Jul 21 17:16:40.645740 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.645720 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 21 17:16:40.732409 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.732361 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60057d50-b81d-40c7-b5e3-9e84050370f9-tls-certs\") pod \"model-serving-api-54d89bd5fc-fpb9n\" (UID: \"60057d50-b81d-40c7-b5e3-9e84050370f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:40.732409 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.732412 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9k26\" (UniqueName: \"kubernetes.io/projected/60057d50-b81d-40c7-b5e3-9e84050370f9-kube-api-access-p9k26\") pod \"model-serving-api-54d89bd5fc-fpb9n\" (UID: \"60057d50-b81d-40c7-b5e3-9e84050370f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:40.833092 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.833055 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60057d50-b81d-40c7-b5e3-9e84050370f9-tls-certs\") pod \"model-serving-api-54d89bd5fc-fpb9n\" (UID: \"60057d50-b81d-40c7-b5e3-9e84050370f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:40.833092 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.833093 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9k26\" (UniqueName: \"kubernetes.io/projected/60057d50-b81d-40c7-b5e3-9e84050370f9-kube-api-access-p9k26\") pod \"model-serving-api-54d89bd5fc-fpb9n\" (UID: \"60057d50-b81d-40c7-b5e3-9e84050370f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:40.835899 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.835862 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/60057d50-b81d-40c7-b5e3-9e84050370f9-tls-certs\") pod \"model-serving-api-54d89bd5fc-fpb9n\" (UID: \"60057d50-b81d-40c7-b5e3-9e84050370f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:40.842579 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.842545 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9k26\" (UniqueName: \"kubernetes.io/projected/60057d50-b81d-40c7-b5e3-9e84050370f9-kube-api-access-p9k26\") pod \"model-serving-api-54d89bd5fc-fpb9n\" (UID: \"60057d50-b81d-40c7-b5e3-9e84050370f9\") " pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:40.954544 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:40.954441 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:41.096214 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:41.096183 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-fpb9n"] Jul 21 17:16:41.098714 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:16:41.098684 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60057d50_b81d_40c7_b5e3_9e84050370f9.slice/crio-9626cd6e5f9cf0af5797dbb528359137bab8c0c18c85d163f71a31af0c9e9e39 WatchSource:0}: Error finding container 9626cd6e5f9cf0af5797dbb528359137bab8c0c18c85d163f71a31af0c9e9e39: Status 404 returned error can't find the container with id 9626cd6e5f9cf0af5797dbb528359137bab8c0c18c85d163f71a31af0c9e9e39 Jul 21 17:16:41.742946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:41.742908 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" event={"ID":"60057d50-b81d-40c7-b5e3-9e84050370f9","Type":"ContainerStarted","Data":"9626cd6e5f9cf0af5797dbb528359137bab8c0c18c85d163f71a31af0c9e9e39"} Jul 21 17:16:43.751682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:43.751641 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" event={"ID":"60057d50-b81d-40c7-b5e3-9e84050370f9","Type":"ContainerStarted","Data":"4ebd1eb60c980e67a1e1206dc3c3881806b07fa13965007f0d61f020d550d114"} Jul 21 17:16:43.752103 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:43.751790 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:16:43.770912 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:43.770856 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" podStartSLOduration=1.447602861 podStartE2EDuration="3.770837998s" totalImagesPullingTime="2.323235137s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:16:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:16:43.769322585 +0000 UTC m=+503.382866888" watchObservedRunningTime="2026-07-21 17:16:43.770837998 +0000 UTC m=+503.384382298" Jul 21 17:16:46.808993 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.808949 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s"] Jul 21 17:16:46.812079 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.812058 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.814879 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.814845 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 21 17:16:46.815660 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.815639 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 21 17:16:46.815813 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.815795 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-7p6s7\"" Jul 21 17:16:46.815876 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.815827 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 21 17:16:46.825833 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.825809 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s"] Jul 21 17:16:46.880408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880370 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880404 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880630 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880429 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880630 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880452 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880630 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880492 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880630 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880516 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880630 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880535 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880630 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880612 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.880821 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.880655 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbq4j\" (UniqueName: \"kubernetes.io/projected/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-kube-api-access-sbq4j\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981069 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981090 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981114 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981204 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981265 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbq4j\" (UniqueName: \"kubernetes.io/projected/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-kube-api-access-sbq4j\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981294 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981319 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981719 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981561 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981719 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981677 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981914 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981885 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.981979 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.981947 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.982026 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.982008 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.983672 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.983652 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.983904 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.983882 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:46.992623 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:46.992593 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:47.003190 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.003164 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbq4j\" (UniqueName: \"kubernetes.io/projected/fde8b9dd-5666-4d6e-9d9a-f9791a014f1f-kube-api-access-sbq4j\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4lgh9s\" (UID: \"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:47.122678 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.122583 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:47.271345 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.271305 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s"] Jul 21 17:16:47.277146 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:16:47.277121 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde8b9dd_5666_4d6e_9d9a_f9791a014f1f.slice/crio-a5fbe933e633a1e97d41321cb8abd82970c169e8e0912a7efbf61ae27b69ed1f WatchSource:0}: Error finding container a5fbe933e633a1e97d41321cb8abd82970c169e8e0912a7efbf61ae27b69ed1f: Status 404 returned error can't find the container with id a5fbe933e633a1e97d41321cb8abd82970c169e8e0912a7efbf61ae27b69ed1f Jul 21 17:16:47.323517 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.323479 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs"] Jul 21 17:16:47.327938 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.327911 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.331878 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.331856 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 21 17:16:47.332386 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.332367 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 21 17:16:47.332537 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.332518 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 21 17:16:47.332605 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.332564 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-kzszl\"" Jul 21 17:16:47.343416 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.343385 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs"] Jul 21 17:16:47.384920 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.384835 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a21adf3-2640-4112-857b-740b6f155b60-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.384920 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.384872 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.384920 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.384897 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.385155 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.384989 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a21adf3-2640-4112-857b-740b6f155b60-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.385155 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.385041 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-znlpm\" (UniqueName: \"kubernetes.io/projected/8a21adf3-2640-4112-857b-740b6f155b60-kube-api-access-znlpm\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.385155 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.385074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a21adf3-2640-4112-857b-740b6f155b60-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.385155 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.385093 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.485682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.485641 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a21adf3-2640-4112-857b-740b6f155b60-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.485682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.485680 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.485913 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.485704 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.485913 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.485839 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a21adf3-2640-4112-857b-740b6f155b60-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.485913 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.485882 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-znlpm\" (UniqueName: \"kubernetes.io/projected/8a21adf3-2640-4112-857b-740b6f155b60-kube-api-access-znlpm\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.486038 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.485925 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a21adf3-2640-4112-857b-740b6f155b60-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.486038 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.485954 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.486725 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.486682 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8a21adf3-2640-4112-857b-740b6f155b60-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.488221 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.488197 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a21adf3-2640-4112-857b-740b6f155b60-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.488384 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.488368 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.488532 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.488512 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.488612 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.488592 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8a21adf3-2640-4112-857b-740b6f155b60-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.506636 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.506598 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a21adf3-2640-4112-857b-740b6f155b60-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.506772 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.506667 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-znlpm\" (UniqueName: \"kubernetes.io/projected/8a21adf3-2640-4112-857b-740b6f155b60-kube-api-access-znlpm\") pod \"istiod-openshift-gateway-94bb8fbfd-2v9bs\" (UID: \"8a21adf3-2640-4112-857b-740b6f155b60\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.637970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.637876 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:47.766754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.766719 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" event={"ID":"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f","Type":"ContainerStarted","Data":"a5fbe933e633a1e97d41321cb8abd82970c169e8e0912a7efbf61ae27b69ed1f"} Jul 21 17:16:47.772171 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:47.772142 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs"] Jul 21 17:16:47.775525 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:16:47.775497 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a21adf3_2640_4112_857b_740b6f155b60.slice/crio-9fd52471fadee104bda267ae90807f2fa853eeaf69d44d2c46f203edb7d9fe86 WatchSource:0}: Error finding container 9fd52471fadee104bda267ae90807f2fa853eeaf69d44d2c46f203edb7d9fe86: Status 404 returned error can't find the container with id 9fd52471fadee104bda267ae90807f2fa853eeaf69d44d2c46f203edb7d9fe86 Jul 21 17:16:48.773742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:48.773695 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" event={"ID":"8a21adf3-2640-4112-857b-740b6f155b60","Type":"ContainerStarted","Data":"9fd52471fadee104bda267ae90807f2fa853eeaf69d44d2c46f203edb7d9fe86"} Jul 21 17:16:51.119658 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.119624 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:16:51.119957 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.119699 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:16:51.119957 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.119728 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:16:51.120525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.120494 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:16:51.120599 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.120580 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:16:51.788362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.788326 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" event={"ID":"8a21adf3-2640-4112-857b-740b6f155b60","Type":"ContainerStarted","Data":"6a4d30bac315f0a433cd0d3ebda6425dac65fb6f2e14979ac0b9b6903e1fedd5"} Jul 21 17:16:51.788560 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.788458 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:51.789804 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.789780 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" event={"ID":"fde8b9dd-5666-4d6e-9d9a-f9791a014f1f","Type":"ContainerStarted","Data":"c8f559246231181717ef9590ebfaffe0a8cc8267d69d42b1531e347255ce4f1c"} Jul 21 17:16:51.824170 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.824101 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" podStartSLOduration=1.481465934 podStartE2EDuration="4.824079788s" totalImagesPullingTime="3.342613854s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:16:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:16:51.822723382 +0000 UTC m=+511.436267683" watchObservedRunningTime="2026-07-21 17:16:51.824079788 +0000 UTC m=+511.437624092" Jul 21 17:16:51.884173 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:51.884106 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" podStartSLOduration=2.043862604 podStartE2EDuration="5.88408215s" totalImagesPullingTime="3.840219546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:16:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:16:51.881402543 +0000 UTC m=+511.494946846" watchObservedRunningTime="2026-07-21 17:16:51.88408215 +0000 UTC m=+511.497626454" Jul 21 17:16:52.123357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:52.123262 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:52.127824 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:52.127804 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:52.794250 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:52.794190 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:52.795174 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:52.795157 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4lgh9s" Jul 21 17:16:52.795868 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:52.795848 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-2v9bs" Jul 21 17:16:54.759754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:16:54.759726 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-fpb9n" Jul 21 17:19:48.190766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.190729 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v"] Jul 21 17:19:48.193959 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.193943 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.196519 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.196436 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 21 17:19:48.196519 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.196439 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 21 17:19:48.196519 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.196439 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-tbb7w\"" Jul 21 17:19:48.197317 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.197303 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 21 17:19:48.203345 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.203326 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v"] Jul 21 17:19:48.232112 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.232083 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4075c18e-1137-4749-859f-2b30253d2fa0-data\") pod \"postgres-69fb4594cf-kzh2v\" (UID: \"4075c18e-1137-4749-859f-2b30253d2fa0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.232273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.232127 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnpjq\" (UniqueName: \"kubernetes.io/projected/4075c18e-1137-4749-859f-2b30253d2fa0-kube-api-access-fnpjq\") pod \"postgres-69fb4594cf-kzh2v\" (UID: \"4075c18e-1137-4749-859f-2b30253d2fa0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.332870 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.332832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnpjq\" (UniqueName: \"kubernetes.io/projected/4075c18e-1137-4749-859f-2b30253d2fa0-kube-api-access-fnpjq\") pod \"postgres-69fb4594cf-kzh2v\" (UID: \"4075c18e-1137-4749-859f-2b30253d2fa0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.333061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.332920 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4075c18e-1137-4749-859f-2b30253d2fa0-data\") pod \"postgres-69fb4594cf-kzh2v\" (UID: \"4075c18e-1137-4749-859f-2b30253d2fa0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.333339 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.333318 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4075c18e-1137-4749-859f-2b30253d2fa0-data\") pod \"postgres-69fb4594cf-kzh2v\" (UID: \"4075c18e-1137-4749-859f-2b30253d2fa0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.341720 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.341700 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnpjq\" (UniqueName: \"kubernetes.io/projected/4075c18e-1137-4749-859f-2b30253d2fa0-kube-api-access-fnpjq\") pod \"postgres-69fb4594cf-kzh2v\" (UID: \"4075c18e-1137-4749-859f-2b30253d2fa0\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.504359 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.504332 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:48.629544 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:48.629515 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v"] Jul 21 17:19:48.632314 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:19:48.632287 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4075c18e_1137_4749_859f_2b30253d2fa0.slice/crio-6d9af35a1f5793759b0456b6450bb54e609d340bdadefd642898a7e706714384 WatchSource:0}: Error finding container 6d9af35a1f5793759b0456b6450bb54e609d340bdadefd642898a7e706714384: Status 404 returned error can't find the container with id 6d9af35a1f5793759b0456b6450bb54e609d340bdadefd642898a7e706714384 Jul 21 17:19:49.435942 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:49.435903 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" event={"ID":"4075c18e-1137-4749-859f-2b30253d2fa0","Type":"ContainerStarted","Data":"6d9af35a1f5793759b0456b6450bb54e609d340bdadefd642898a7e706714384"} Jul 21 17:19:54.460314 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:54.460282 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" event={"ID":"4075c18e-1137-4749-859f-2b30253d2fa0","Type":"ContainerStarted","Data":"c2c81a486cfdbdb03830ce58aa8e57e41e1bb8867db4eaa9b8e34b8df25da2e9"} Jul 21 17:19:54.460676 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:54.460412 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:19:54.477452 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:19:54.477408 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" podStartSLOduration=1.310723482 podStartE2EDuration="6.477393448s" totalImagesPullingTime="5.166669966s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:19:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:19:54.476599144 +0000 UTC m=+694.090143445" watchObservedRunningTime="2026-07-21 17:19:54.477393448 +0000 UTC m=+694.090937749" Jul 21 17:20:00.492712 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:20:00.492684 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-kzh2v" Jul 21 17:21:44.320090 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.319998 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn"] Jul 21 17:21:44.323481 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.323457 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.326433 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.326410 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 21 17:21:44.327284 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.327256 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 21 17:21:44.327284 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.327275 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 21 17:21:44.327443 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.327299 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-78xwk\"" Jul 21 17:21:44.332119 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.332094 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn"] Jul 21 17:21:44.365197 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.365153 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.365406 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.365215 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.365406 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.365265 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.365406 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.365301 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.365406 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.365332 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.365406 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.365373 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p4hs\" (UniqueName: \"kubernetes.io/projected/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-kube-api-access-6p4hs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466227 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466471 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466335 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p4hs\" (UniqueName: \"kubernetes.io/projected/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-kube-api-access-6p4hs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466471 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466471 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466427 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466471 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466675 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466484 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466731 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466696 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466799 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466783 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.466867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.466846 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.468555 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.468533 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.469025 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.469005 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.474984 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.474959 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p4hs\" (UniqueName: \"kubernetes.io/projected/d6fed8dd-4022-47d3-aeeb-0811dab4b48a-kube-api-access-6p4hs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rnkwn\" (UID: \"d6fed8dd-4022-47d3-aeeb-0811dab4b48a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.634493 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.634389 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:44.772216 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.772192 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn"] Jul 21 17:21:44.774231 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:21:44.774199 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6fed8dd_4022_47d3_aeeb_0811dab4b48a.slice/crio-c23b7e18d015a233cdf5c55d5b21a6b207baff2d2056342da0746a0f128b0519 WatchSource:0}: Error finding container c23b7e18d015a233cdf5c55d5b21a6b207baff2d2056342da0746a0f128b0519: Status 404 returned error can't find the container with id c23b7e18d015a233cdf5c55d5b21a6b207baff2d2056342da0746a0f128b0519 Jul 21 17:21:44.775618 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.775602 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 21 17:21:44.887558 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:44.887466 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" event={"ID":"d6fed8dd-4022-47d3-aeeb-0811dab4b48a","Type":"ContainerStarted","Data":"c23b7e18d015a233cdf5c55d5b21a6b207baff2d2056342da0746a0f128b0519"} Jul 21 17:21:45.893494 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:45.893457 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" event={"ID":"d6fed8dd-4022-47d3-aeeb-0811dab4b48a","Type":"ContainerStarted","Data":"367512027de8ce8b3fb4a2250ff17e0b84b5245d3c8d299b997fc98247140a65"} Jul 21 17:21:45.914545 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:45.914496 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" podStartSLOduration=0.885818669 podStartE2EDuration="1.914477092s" totalImagesPullingTime="1.028658423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:21:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:21:45.912721456 +0000 UTC m=+805.526265757" watchObservedRunningTime="2026-07-21 17:21:45.914477092 +0000 UTC m=+805.528021393" Jul 21 17:21:54.634639 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:54.634595 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:54.635123 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:54.634704 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:54.646924 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:54.646898 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:21:54.941259 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:21:54.941138 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rnkwn" Jul 21 17:33:54.029851 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.029775 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk"] Jul 21 17:33:54.035387 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.034327 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.037505 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.037479 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-872e5f75-kserve-self-signed-certs\"" Jul 21 17:33:54.058942 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.058920 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk"] Jul 21 17:33:54.062682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.062657 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-model-cache\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.062780 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.062715 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvbj8\" (UniqueName: \"kubernetes.io/projected/61f7a4ab-4faf-4684-930e-71bd908e3c01-kube-api-access-tvbj8\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.062780 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.062774 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f7a4ab-4faf-4684-930e-71bd908e3c01-tls-certs\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.062863 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.062835 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-dshm\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.062921 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.062901 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-tmp-dir\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.062980 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.062964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-home\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.163742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.163710 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f7a4ab-4faf-4684-930e-71bd908e3c01-tls-certs\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.163933 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.163753 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-dshm\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.163933 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.163773 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-tmp-dir\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.163933 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.163795 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-home\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.163933 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.163919 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-model-cache\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.164699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.163982 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvbj8\" (UniqueName: \"kubernetes.io/projected/61f7a4ab-4faf-4684-930e-71bd908e3c01-kube-api-access-tvbj8\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.164699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.164220 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-home\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.164699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.164298 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-tmp-dir\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.164699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.164405 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-model-cache\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.166104 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.166075 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-dshm\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.166514 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.166495 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f7a4ab-4faf-4684-930e-71bd908e3c01-tls-certs\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.173029 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.173004 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvbj8\" (UniqueName: \"kubernetes.io/projected/61f7a4ab-4faf-4684-930e-71bd908e3c01-kube-api-access-tvbj8\") pod \"e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.343897 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.343807 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:33:54.470213 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.470042 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk"] Jul 21 17:33:54.472252 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:33:54.472211 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61f7a4ab_4faf_4684_930e_71bd908e3c01.slice/crio-d82496e0756bbed0f3203d528bb8a86065be4cf9031464c136a25c4620f0347d WatchSource:0}: Error finding container d82496e0756bbed0f3203d528bb8a86065be4cf9031464c136a25c4620f0347d: Status 404 returned error can't find the container with id d82496e0756bbed0f3203d528bb8a86065be4cf9031464c136a25c4620f0347d Jul 21 17:33:54.745701 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.745666 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" event={"ID":"61f7a4ab-4faf-4684-930e-71bd908e3c01","Type":"ContainerStarted","Data":"f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987"} Jul 21 17:33:54.745889 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.745709 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" event={"ID":"61f7a4ab-4faf-4684-930e-71bd908e3c01","Type":"ContainerStarted","Data":"d82496e0756bbed0f3203d528bb8a86065be4cf9031464c136a25c4620f0347d"} Jul 21 17:33:54.768656 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:33:54.768607 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" podStartSLOduration=0.768593212 podStartE2EDuration="768.593212ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:33:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:33:54.768220993 +0000 UTC m=+1534.381765296" watchObservedRunningTime="2026-07-21 17:33:54.768593212 +0000 UTC m=+1534.382137511" Jul 21 17:34:00.927178 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:00.927146 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r"] Jul 21 17:34:00.930404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:00.930388 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:00.933362 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:00.933341 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-872e5f75-kserve-self-signed-certs\"" Jul 21 17:34:00.941071 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:00.941052 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r"] Jul 21 17:34:01.012909 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.012883 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-model-cache\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.013069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.012922 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-home\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.013069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.012999 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82965792-7914-4e69-9624-2b73f5279647-tls-certs\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.013069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.013035 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkj26\" (UniqueName: \"kubernetes.io/projected/82965792-7914-4e69-9624-2b73f5279647-kube-api-access-wkj26\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.013069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.013056 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-dshm\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.013202 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.013129 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-tmp-dir\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114331 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82965792-7914-4e69-9624-2b73f5279647-tls-certs\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114331 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114337 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkj26\" (UniqueName: \"kubernetes.io/projected/82965792-7914-4e69-9624-2b73f5279647-kube-api-access-wkj26\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114566 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114358 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-dshm\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114566 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114402 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-tmp-dir\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114566 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114426 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-model-cache\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114566 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114455 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-home\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114845 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-model-cache\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114869 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-home\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.114987 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.114958 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-tmp-dir\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.116711 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.116689 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-dshm\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.116890 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.116872 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82965792-7914-4e69-9624-2b73f5279647-tls-certs\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.127782 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.127758 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkj26\" (UniqueName: \"kubernetes.io/projected/82965792-7914-4e69-9624-2b73f5279647-kube-api-access-wkj26\") pod \"e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.239172 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.239094 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:01.377572 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.377547 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r"] Jul 21 17:34:01.379913 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:34:01.379886 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82965792_7914_4e69_9624_2b73f5279647.slice/crio-63912764dd916e40076817dd693b72ddac204b7a6bec2509f7218e9fd26f9e8c WatchSource:0}: Error finding container 63912764dd916e40076817dd693b72ddac204b7a6bec2509f7218e9fd26f9e8c: Status 404 returned error can't find the container with id 63912764dd916e40076817dd693b72ddac204b7a6bec2509f7218e9fd26f9e8c Jul 21 17:34:01.775197 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.775164 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" event={"ID":"82965792-7914-4e69-9624-2b73f5279647","Type":"ContainerStarted","Data":"ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6"} Jul 21 17:34:01.775197 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.775202 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" event={"ID":"82965792-7914-4e69-9624-2b73f5279647","Type":"ContainerStarted","Data":"63912764dd916e40076817dd693b72ddac204b7a6bec2509f7218e9fd26f9e8c"} Jul 21 17:34:01.798404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:01.798358 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" podStartSLOduration=1.7983426919999999 podStartE2EDuration="1.798342692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:34:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:34:01.796320425 +0000 UTC m=+1541.409864727" watchObservedRunningTime="2026-07-21 17:34:01.798342692 +0000 UTC m=+1541.411886993" Jul 21 17:34:04.344974 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:04.344886 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:34:04.344974 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:04.344937 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:34:04.357348 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:04.357328 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:34:04.799181 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:04.799152 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:34:11.240087 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:11.240054 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:11.240605 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:11.240106 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:11.252505 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:11.252478 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:11.828385 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:11.828357 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:32.899299 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:32.899265 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk"] Jul 21 17:34:34.013938 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:34.013904 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r"] Jul 21 17:34:47.918777 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:47.918714 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" podUID="61f7a4ab-4faf-4684-930e-71bd908e3c01" containerName="main" containerID="cri-o://f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987" gracePeriod=45 Jul 21 17:34:48.169395 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.169330 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:34:48.208684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.208649 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvbj8\" (UniqueName: \"kubernetes.io/projected/61f7a4ab-4faf-4684-930e-71bd908e3c01-kube-api-access-tvbj8\") pod \"61f7a4ab-4faf-4684-930e-71bd908e3c01\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " Jul 21 17:34:48.208871 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.208691 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-model-cache\") pod \"61f7a4ab-4faf-4684-930e-71bd908e3c01\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " Jul 21 17:34:48.208871 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.208731 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-home\") pod \"61f7a4ab-4faf-4684-930e-71bd908e3c01\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " Jul 21 17:34:48.208871 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.208755 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-tmp-dir\") pod \"61f7a4ab-4faf-4684-930e-71bd908e3c01\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " Jul 21 17:34:48.208871 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.208783 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f7a4ab-4faf-4684-930e-71bd908e3c01-tls-certs\") pod \"61f7a4ab-4faf-4684-930e-71bd908e3c01\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " Jul 21 17:34:48.208871 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.208800 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-dshm\") pod \"61f7a4ab-4faf-4684-930e-71bd908e3c01\" (UID: \"61f7a4ab-4faf-4684-930e-71bd908e3c01\") " Jul 21 17:34:48.209100 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.208994 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-model-cache" (OuterVolumeSpecName: "model-cache") pod "61f7a4ab-4faf-4684-930e-71bd908e3c01" (UID: "61f7a4ab-4faf-4684-930e-71bd908e3c01"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:48.209100 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.209004 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-home" (OuterVolumeSpecName: "home") pod "61f7a4ab-4faf-4684-930e-71bd908e3c01" (UID: "61f7a4ab-4faf-4684-930e-71bd908e3c01"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:48.209100 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.209048 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "61f7a4ab-4faf-4684-930e-71bd908e3c01" (UID: "61f7a4ab-4faf-4684-930e-71bd908e3c01"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:48.210923 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.210890 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-dshm" (OuterVolumeSpecName: "dshm") pod "61f7a4ab-4faf-4684-930e-71bd908e3c01" (UID: "61f7a4ab-4faf-4684-930e-71bd908e3c01"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:48.211030 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.211011 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61f7a4ab-4faf-4684-930e-71bd908e3c01-kube-api-access-tvbj8" (OuterVolumeSpecName: "kube-api-access-tvbj8") pod "61f7a4ab-4faf-4684-930e-71bd908e3c01" (UID: "61f7a4ab-4faf-4684-930e-71bd908e3c01"). InnerVolumeSpecName "kube-api-access-tvbj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:34:48.211080 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.211030 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61f7a4ab-4faf-4684-930e-71bd908e3c01-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "61f7a4ab-4faf-4684-930e-71bd908e3c01" (UID: "61f7a4ab-4faf-4684-930e-71bd908e3c01"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:34:48.309516 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.309481 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-home\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:48.309516 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.309509 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-tmp-dir\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:48.309516 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.309519 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61f7a4ab-4faf-4684-930e-71bd908e3c01-tls-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:48.309516 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.309527 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-dshm\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:48.309782 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.309537 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvbj8\" (UniqueName: \"kubernetes.io/projected/61f7a4ab-4faf-4684-930e-71bd908e3c01-kube-api-access-tvbj8\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:48.309782 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.309546 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61f7a4ab-4faf-4684-930e-71bd908e3c01-model-cache\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:48.965325 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.965288 2565 generic.go:356] "Generic (PLEG): container finished" podID="61f7a4ab-4faf-4684-930e-71bd908e3c01" containerID="f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987" exitCode=0 Jul 21 17:34:48.965728 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.965340 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" Jul 21 17:34:48.965728 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.965362 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" event={"ID":"61f7a4ab-4faf-4684-930e-71bd908e3c01","Type":"ContainerDied","Data":"f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987"} Jul 21 17:34:48.965728 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.965401 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk" event={"ID":"61f7a4ab-4faf-4684-930e-71bd908e3c01","Type":"ContainerDied","Data":"d82496e0756bbed0f3203d528bb8a86065be4cf9031464c136a25c4620f0347d"} Jul 21 17:34:48.965728 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.965422 2565 scope.go:117] "RemoveContainer" containerID="f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987" Jul 21 17:34:48.974347 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.974327 2565 scope.go:117] "RemoveContainer" containerID="f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987" Jul 21 17:34:48.974635 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:34:48.974612 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987\": container with ID starting with f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987 not found: ID does not exist" containerID="f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987" Jul 21 17:34:48.974703 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.974645 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987"} err="failed to get container status \"f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987\": rpc error: code = NotFound desc = could not find container \"f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987\": container with ID starting with f457cb4bfae134566cd7ae028230f10fd07a6f0e9f32ee16faa9c5786a40c987 not found: ID does not exist" Jul 21 17:34:48.990338 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.990310 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk"] Jul 21 17:34:48.994486 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:48.994462 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-872e5f75-kserve-654559fcbc-ft7xk"] Jul 21 17:34:49.032604 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.032571 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61f7a4ab-4faf-4684-930e-71bd908e3c01" path="/var/lib/kubelet/pods/61f7a4ab-4faf-4684-930e-71bd908e3c01/volumes" Jul 21 17:34:49.033143 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.033116 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" podUID="82965792-7914-4e69-9624-2b73f5279647" containerName="main" containerID="cri-o://ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6" gracePeriod=45 Jul 21 17:34:49.275810 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.275785 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:49.323826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.323786 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wkj26\" (UniqueName: \"kubernetes.io/projected/82965792-7914-4e69-9624-2b73f5279647-kube-api-access-wkj26\") pod \"82965792-7914-4e69-9624-2b73f5279647\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " Jul 21 17:34:49.323826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.323840 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82965792-7914-4e69-9624-2b73f5279647-tls-certs\") pod \"82965792-7914-4e69-9624-2b73f5279647\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " Jul 21 17:34:49.324074 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.323864 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-home\") pod \"82965792-7914-4e69-9624-2b73f5279647\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " Jul 21 17:34:49.324074 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.323973 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-model-cache\") pod \"82965792-7914-4e69-9624-2b73f5279647\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " Jul 21 17:34:49.324074 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.324013 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-tmp-dir\") pod \"82965792-7914-4e69-9624-2b73f5279647\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " Jul 21 17:34:49.324074 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.324050 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-dshm\") pod \"82965792-7914-4e69-9624-2b73f5279647\" (UID: \"82965792-7914-4e69-9624-2b73f5279647\") " Jul 21 17:34:49.324320 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.324095 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-home" (OuterVolumeSpecName: "home") pod "82965792-7914-4e69-9624-2b73f5279647" (UID: "82965792-7914-4e69-9624-2b73f5279647"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:49.324320 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.324279 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-model-cache" (OuterVolumeSpecName: "model-cache") pod "82965792-7914-4e69-9624-2b73f5279647" (UID: "82965792-7914-4e69-9624-2b73f5279647"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:49.324415 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.324340 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-home\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:49.324415 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.324354 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-model-cache\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:49.324415 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.324349 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "82965792-7914-4e69-9624-2b73f5279647" (UID: "82965792-7914-4e69-9624-2b73f5279647"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:49.325963 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.325940 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-dshm" (OuterVolumeSpecName: "dshm") pod "82965792-7914-4e69-9624-2b73f5279647" (UID: "82965792-7914-4e69-9624-2b73f5279647"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:34:49.326061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.325978 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82965792-7914-4e69-9624-2b73f5279647-kube-api-access-wkj26" (OuterVolumeSpecName: "kube-api-access-wkj26") pod "82965792-7914-4e69-9624-2b73f5279647" (UID: "82965792-7914-4e69-9624-2b73f5279647"). InnerVolumeSpecName "kube-api-access-wkj26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:34:49.326061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.326008 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82965792-7914-4e69-9624-2b73f5279647-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "82965792-7914-4e69-9624-2b73f5279647" (UID: "82965792-7914-4e69-9624-2b73f5279647"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:34:49.425500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.425451 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-dshm\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:49.425500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.425495 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wkj26\" (UniqueName: \"kubernetes.io/projected/82965792-7914-4e69-9624-2b73f5279647-kube-api-access-wkj26\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:49.425500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.425507 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82965792-7914-4e69-9624-2b73f5279647-tls-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:49.425500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.425516 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82965792-7914-4e69-9624-2b73f5279647-tmp-dir\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:34:49.971399 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.971369 2565 generic.go:356] "Generic (PLEG): container finished" podID="82965792-7914-4e69-9624-2b73f5279647" containerID="ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6" exitCode=0 Jul 21 17:34:49.971801 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.971409 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" Jul 21 17:34:49.971801 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.971424 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" event={"ID":"82965792-7914-4e69-9624-2b73f5279647","Type":"ContainerDied","Data":"ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6"} Jul 21 17:34:49.971801 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.971457 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r" event={"ID":"82965792-7914-4e69-9624-2b73f5279647","Type":"ContainerDied","Data":"63912764dd916e40076817dd693b72ddac204b7a6bec2509f7218e9fd26f9e8c"} Jul 21 17:34:49.971801 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.971474 2565 scope.go:117] "RemoveContainer" containerID="ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6" Jul 21 17:34:49.979845 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.979827 2565 scope.go:117] "RemoveContainer" containerID="ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6" Jul 21 17:34:49.980137 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:34:49.980119 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6\": container with ID starting with ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6 not found: ID does not exist" containerID="ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6" Jul 21 17:34:49.980184 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.980146 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6"} err="failed to get container status \"ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6\": rpc error: code = NotFound desc = could not find container \"ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6\": container with ID starting with ab1b1376d36f269bf5d0ddde8f0ac2499b9d28f1f7dffc09841546a788081cb6 not found: ID does not exist" Jul 21 17:34:49.994233 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.994209 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r"] Jul 21 17:34:49.999076 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:49.999055 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-872e5f75-kserve-59b79dc947-nbm6r"] Jul 21 17:34:51.033162 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:34:51.033124 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82965792-7914-4e69-9624-2b73f5279647" path="/var/lib/kubelet/pods/82965792-7914-4e69-9624-2b73f5279647/volumes" Jul 21 17:38:11.998211 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:11.998120 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf"] Jul 21 17:38:12.000668 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:11.998551 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61f7a4ab-4faf-4684-930e-71bd908e3c01" containerName="main" Jul 21 17:38:12.000668 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:11.998569 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="61f7a4ab-4faf-4684-930e-71bd908e3c01" containerName="main" Jul 21 17:38:12.000668 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:11.998581 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82965792-7914-4e69-9624-2b73f5279647" containerName="main" Jul 21 17:38:12.000668 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:11.998589 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="82965792-7914-4e69-9624-2b73f5279647" containerName="main" Jul 21 17:38:12.000668 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:11.998669 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="82965792-7914-4e69-9624-2b73f5279647" containerName="main" Jul 21 17:38:12.000668 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:11.998678 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="61f7a4ab-4faf-4684-930e-71bd908e3c01" containerName="main" Jul 21 17:38:12.001540 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.001521 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.004011 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.003993 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-01e22ac5-openshift-default-dockercfg-7g6pb\"" Jul 21 17:38:12.023297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.023271 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf"] Jul 21 17:38:12.031764 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-envoy\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.031898 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031805 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-socket\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.031898 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031843 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-credential-socket\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.031898 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031871 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpws7\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-kube-api-access-zpws7\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.032072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031912 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-podinfo\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.032072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031928 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-token\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.032072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031974 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-data\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.032072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.031997 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73bfd29-9828-46f2-810b-1de5f32f8fff-istiod-ca-cert\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.032072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.032015 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-certs\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.132996 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.132971 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-credential-socket\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133169 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133029 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpws7\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-kube-api-access-zpws7\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133169 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-podinfo\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133169 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133075 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-token\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133169 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-data\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133169 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133120 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73bfd29-9828-46f2-810b-1de5f32f8fff-istiod-ca-cert\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133169 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133149 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-certs\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133478 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-envoy\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133478 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133288 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-socket\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133571 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133508 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-credential-socket\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133685 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133659 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-certs\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133854 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133830 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-socket\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133854 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133847 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-data\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.133961 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.133924 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73bfd29-9828-46f2-810b-1de5f32f8fff-istiod-ca-cert\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.135634 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.135616 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-envoy\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.135699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.135653 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-podinfo\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.141114 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.141089 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-token\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.142154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.142132 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpws7\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-kube-api-access-zpws7\") pod \"e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.311275 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.311179 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:12.442978 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.442946 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf"] Jul 21 17:38:12.446762 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:38:12.446735 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode73bfd29_9828_46f2_810b_1de5f32f8fff.slice/crio-07d0ed9dd79ef8bbd07133260e2cc1fddf60d790f14d274b83fb4c4d623d578d WatchSource:0}: Error finding container 07d0ed9dd79ef8bbd07133260e2cc1fddf60d790f14d274b83fb4c4d623d578d: Status 404 returned error can't find the container with id 07d0ed9dd79ef8bbd07133260e2cc1fddf60d790f14d274b83fb4c4d623d578d Jul 21 17:38:12.448946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.448912 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:12.449015 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.448989 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:12.449067 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.449052 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:12.771120 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.771084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" event={"ID":"e73bfd29-9828-46f2-810b-1de5f32f8fff","Type":"ContainerStarted","Data":"5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128"} Jul 21 17:38:12.771340 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.771140 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" event={"ID":"e73bfd29-9828-46f2-810b-1de5f32f8fff","Type":"ContainerStarted","Data":"07d0ed9dd79ef8bbd07133260e2cc1fddf60d790f14d274b83fb4c4d623d578d"} Jul 21 17:38:12.793804 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:12.793759 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" podStartSLOduration=1.793745113 podStartE2EDuration="1.793745113s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:38:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:38:12.791495591 +0000 UTC m=+1792.405039891" watchObservedRunningTime="2026-07-21 17:38:12.793745113 +0000 UTC m=+1792.407289413" Jul 21 17:38:13.311853 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:13.311823 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:13.316606 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:13.316582 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:13.775701 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:13.775672 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:13.776699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:13.776681 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:24.828626 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:24.828532 2565 generic.go:356] "Generic (PLEG): container finished" podID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerID="5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128" exitCode=0 Jul 21 17:38:24.828626 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:24.828576 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" event={"ID":"e73bfd29-9828-46f2-810b-1de5f32f8fff","Type":"ContainerDied","Data":"5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128"} Jul 21 17:38:24.829202 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:24.829047 2565 scope.go:117] "RemoveContainer" containerID="5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128" Jul 21 17:38:24.830186 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:24.830157 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:24.830276 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:24.830222 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:24.830276 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:24.830272 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:25.834868 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:25.834834 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" event={"ID":"e73bfd29-9828-46f2-810b-1de5f32f8fff","Type":"ContainerStarted","Data":"371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e"} Jul 21 17:38:26.311532 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:26.311504 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:26.312662 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:26.312631 2565 patch_prober.go:28] interesting pod/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 21 17:38:26.312791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:26.312694 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 21 17:38:27.311785 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:27.311747 2565 patch_prober.go:28] interesting pod/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 21 17:38:27.312168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:27.311800 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 21 17:38:28.312357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:28.312319 2565 patch_prober.go:28] interesting pod/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 21 17:38:28.312746 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:28.312376 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 21 17:38:28.776417 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:28.776378 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:29.311829 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:29.311792 2565 patch_prober.go:28] interesting pod/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" start-of-body= Jul 21 17:38:29.312008 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:29.311851 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.24:15021/healthz/ready\": dial tcp 10.133.0.24:15021: connect: connection refused" Jul 21 17:38:30.316098 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:30.316069 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:30.317176 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:30.317156 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:38:54.950616 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:54.950587 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf"] Jul 21 17:38:54.951154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:54.950799 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" containerID="cri-o://371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e" gracePeriod=30 Jul 21 17:38:55.903272 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.903218 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m"] Jul 21 17:38:55.906801 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.906779 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.909414 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.909393 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-07f072da-gw-openshift-default-dockercfg-jcn5k\"" Jul 21 17:38:55.919033 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.919006 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m"] Jul 21 17:38:55.994878 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.994830 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00043531-98ba-46f0-a433-a8056a6675e8-istiod-ca-cert\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.994894 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-credential-socket\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.994934 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00043531-98ba-46f0-a433-a8056a6675e8-istio-podinfo\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.994966 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-envoy\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.995019 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-socket\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.995050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-certs\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.995074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-data\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.995119 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-istio-token\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:55.995388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:55.995152 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9bvc\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-kube-api-access-s9bvc\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.095947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.095902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-socket\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.095947 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.095949 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-certs\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.095969 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-data\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096001 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-istio-token\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096026 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9bvc\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-kube-api-access-s9bvc\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096068 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00043531-98ba-46f0-a433-a8056a6675e8-istiod-ca-cert\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-credential-socket\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00043531-98ba-46f0-a433-a8056a6675e8-istio-podinfo\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096222 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096166 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-envoy\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096619 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096433 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-data\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096619 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096470 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-certs\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096619 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096593 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-credential-socket\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.096727 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.096713 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-socket\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.097028 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.097008 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00043531-98ba-46f0-a433-a8056a6675e8-istiod-ca-cert\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.098475 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.098457 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-envoy\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.098791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.098768 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00043531-98ba-46f0-a433-a8056a6675e8-istio-podinfo\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.106569 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.106544 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-istio-token\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.106650 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.106620 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9bvc\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-kube-api-access-s9bvc\") pod \"e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.216827 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.216741 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:56.351111 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.351079 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m"] Jul 21 17:38:56.353834 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:38:56.353812 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00043531_98ba_46f0_a433_a8056a6675e8.slice/crio-23b31efa96dbb1f83c25f57c04979c6c008498dc50c6933e290b55561e22d988 WatchSource:0}: Error finding container 23b31efa96dbb1f83c25f57c04979c6c008498dc50c6933e290b55561e22d988: Status 404 returned error can't find the container with id 23b31efa96dbb1f83c25f57c04979c6c008498dc50c6933e290b55561e22d988 Jul 21 17:38:56.355891 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.355861 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:56.355991 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.355936 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:56.355991 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.355981 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:38:56.966428 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.966392 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" event={"ID":"00043531-98ba-46f0-a433-a8056a6675e8","Type":"ContainerStarted","Data":"ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184"} Jul 21 17:38:56.966620 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.966445 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" event={"ID":"00043531-98ba-46f0-a433-a8056a6675e8","Type":"ContainerStarted","Data":"23b31efa96dbb1f83c25f57c04979c6c008498dc50c6933e290b55561e22d988"} Jul 21 17:38:56.987488 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:56.987425 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" podStartSLOduration=1.987404358 podStartE2EDuration="1.987404358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:38:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:38:56.986261723 +0000 UTC m=+1836.599806016" watchObservedRunningTime="2026-07-21 17:38:56.987404358 +0000 UTC m=+1836.600948665" Jul 21 17:38:57.217720 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:57.217625 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:57.222564 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:57.222541 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:57.970157 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:57.970123 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:38:57.971157 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:38:57.971137 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:39:00.200252 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.200212 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:39:00.335458 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335426 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-socket\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335653 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335475 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-credential-socket\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335653 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335502 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-certs\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335653 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335530 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpws7\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-kube-api-access-zpws7\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335653 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335559 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-podinfo\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335653 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335607 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73bfd29-9828-46f2-810b-1de5f32f8fff-istiod-ca-cert\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335653 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335645 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-data\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335953 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335676 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-envoy\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335953 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335675 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:00.335953 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335703 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-token\") pod \"e73bfd29-9828-46f2-810b-1de5f32f8fff\" (UID: \"e73bfd29-9828-46f2-810b-1de5f32f8fff\") " Jul 21 17:39:00.335953 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335833 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:00.335953 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335849 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:00.336215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.335995 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e73bfd29-9828-46f2-810b-1de5f32f8fff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:39:00.336215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.336025 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.336215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.336051 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.336215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.336065 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.336643 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.336617 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-data" (OuterVolumeSpecName: "istio-data") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:00.337939 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.337916 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-token" (OuterVolumeSpecName: "istio-token") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:39:00.337939 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.337920 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:39:00.338069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.337941 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-kube-api-access-zpws7" (OuterVolumeSpecName: "kube-api-access-zpws7") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "kube-api-access-zpws7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:39:00.338069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.337936 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e73bfd29-9828-46f2-810b-1de5f32f8fff" (UID: "e73bfd29-9828-46f2-810b-1de5f32f8fff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:00.437321 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.437279 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.437321 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.437316 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpws7\" (UniqueName: \"kubernetes.io/projected/e73bfd29-9828-46f2-810b-1de5f32f8fff-kube-api-access-zpws7\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.437321 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.437329 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.437557 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.437341 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e73bfd29-9828-46f2-810b-1de5f32f8fff-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.437557 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.437352 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.437557 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.437362 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e73bfd29-9828-46f2-810b-1de5f32f8fff-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:00.983620 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.983587 2565 generic.go:356] "Generic (PLEG): container finished" podID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerID="371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e" exitCode=0 Jul 21 17:39:00.983799 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.983633 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" Jul 21 17:39:00.983799 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.983635 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" event={"ID":"e73bfd29-9828-46f2-810b-1de5f32f8fff","Type":"ContainerDied","Data":"371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e"} Jul 21 17:39:00.983799 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.983786 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf" event={"ID":"e73bfd29-9828-46f2-810b-1de5f32f8fff","Type":"ContainerDied","Data":"07d0ed9dd79ef8bbd07133260e2cc1fddf60d790f14d274b83fb4c4d623d578d"} Jul 21 17:39:00.983917 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.983818 2565 scope.go:117] "RemoveContainer" containerID="371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e" Jul 21 17:39:00.992489 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.992472 2565 scope.go:117] "RemoveContainer" containerID="5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128" Jul 21 17:39:00.999979 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:00.999962 2565 scope.go:117] "RemoveContainer" containerID="371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e" Jul 21 17:39:01.000254 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:39:01.000226 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e\": container with ID starting with 371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e not found: ID does not exist" containerID="371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e" Jul 21 17:39:01.000293 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:01.000263 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e"} err="failed to get container status \"371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e\": rpc error: code = NotFound desc = could not find container \"371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e\": container with ID starting with 371a03a67b0141d7859cac8d02d491bf004b3a91262e1af738cb0644b0b87d4e not found: ID does not exist" Jul 21 17:39:01.000293 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:01.000279 2565 scope.go:117] "RemoveContainer" containerID="5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128" Jul 21 17:39:01.000502 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:39:01.000486 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128\": container with ID starting with 5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128 not found: ID does not exist" containerID="5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128" Jul 21 17:39:01.000532 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:01.000507 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128"} err="failed to get container status \"5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128\": rpc error: code = NotFound desc = could not find container \"5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128\": container with ID starting with 5e53818eeb32c19c76e608fde6c9e6969523203bb24744d424db5d0e7b68e128 not found: ID does not exist" Jul 21 17:39:01.004983 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:01.004963 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf"] Jul 21 17:39:01.008872 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:01.008851 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-01e22ac5-openshift-default-65cdd659d9-55rnf"] Jul 21 17:39:01.031828 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:01.031802 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" path="/var/lib/kubelet/pods/e73bfd29-9828-46f2-810b-1de5f32f8fff/volumes" Jul 21 17:39:13.215204 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:13.214265 2565 patch_prober.go:28] interesting pod/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": read tcp 10.133.0.2:52584->10.133.0.25:15021: read: connection reset by peer" start-of-body= Jul 21 17:39:13.215204 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:13.214339 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": read tcp 10.133.0.2:52584->10.133.0.25:15021: read: connection reset by peer" Jul 21 17:39:14.037363 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:14.037331 2565 generic.go:356] "Generic (PLEG): container finished" podID="00043531-98ba-46f0-a433-a8056a6675e8" containerID="ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184" exitCode=0 Jul 21 17:39:14.037531 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:14.037390 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" event={"ID":"00043531-98ba-46f0-a433-a8056a6675e8","Type":"ContainerDied","Data":"ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184"} Jul 21 17:39:14.037748 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:14.037734 2565 scope.go:117] "RemoveContainer" containerID="ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184" Jul 21 17:39:14.038823 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:14.038793 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:14.038922 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:14.038878 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:14.038980 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:14.038924 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:15.063216 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:15.063177 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" event={"ID":"00043531-98ba-46f0-a433-a8056a6675e8","Type":"ContainerStarted","Data":"ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b"} Jul 21 17:39:15.217201 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:15.217167 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:39:15.217394 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:15.217370 2565 patch_prober.go:28] interesting pod/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 21 17:39:15.217446 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:15.217422 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 21 17:39:16.217904 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:16.217868 2565 patch_prober.go:28] interesting pod/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 21 17:39:16.218333 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:16.217918 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 21 17:39:17.217809 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:17.217771 2565 patch_prober.go:28] interesting pod/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 21 17:39:17.218012 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:17.217835 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 21 17:39:18.217819 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:18.217777 2565 patch_prober.go:28] interesting pod/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body= Jul 21 17:39:18.218025 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:18.217849 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" Jul 21 17:39:19.220648 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:19.220620 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:39:19.221041 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:19.220866 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:39:19.221569 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:19.221550 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:39:38.744229 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:38.744190 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m"] Jul 21 17:39:38.744862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:38.744420 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" containerID="cri-o://ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b" gracePeriod=30 Jul 21 17:39:39.581561 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.581527 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5"] Jul 21 17:39:39.581813 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.581801 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" Jul 21 17:39:39.581874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.581814 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" Jul 21 17:39:39.581874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.581833 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" Jul 21 17:39:39.581874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.581839 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" Jul 21 17:39:39.581966 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.581887 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" Jul 21 17:39:39.581966 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.581894 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e73bfd29-9828-46f2-810b-1de5f32f8fff" containerName="istio-proxy" Jul 21 17:39:39.584832 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.584816 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.587438 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.587398 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-6b1d8ddb-openshift-default-dockercfg-8bbv8\"" Jul 21 17:39:39.627196 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.627167 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5"] Jul 21 17:39:39.663732 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.663700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-data\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.663869 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.663741 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-credential-socket\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.663869 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.663820 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-envoy\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.663946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.663914 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-podinfo\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.663993 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.663947 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-token\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.664031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.663992 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae5b927b-95ae-4533-b314-e697c7f010d3-istiod-ca-cert\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.664031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.664016 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tg99d\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-kube-api-access-tg99d\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.664110 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.664069 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-socket\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.664110 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.664094 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-certs\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765446 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-socket\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765446 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765448 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-certs\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765482 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-data\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765538 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-credential-socket\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765589 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-envoy\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765780 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-podinfo\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765824 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-token\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765880 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae5b927b-95ae-4533-b314-e697c7f010d3-istiod-ca-cert\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765903 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-data\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tg99d\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-kube-api-access-tg99d\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.765967 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765937 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-credential-socket\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.766367 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.765877 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-certs\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.766367 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.766037 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-socket\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.766534 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.766517 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae5b927b-95ae-4533-b314-e697c7f010d3-istiod-ca-cert\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.767796 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.767777 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-envoy\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.768110 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.768090 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-podinfo\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.777385 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.777359 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-token\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.782672 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.782649 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tg99d\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-kube-api-access-tg99d\") pod \"e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:39.895118 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:39.895040 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:40.030420 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.030388 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5"] Jul 21 17:39:40.033808 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:39:40.033788 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae5b927b_95ae_4533_b314_e697c7f010d3.slice/crio-89cc86b3951d491414dd9cc2f8d9597e7bc0a08a38044abb4492db3925a9cb0a WatchSource:0}: Error finding container 89cc86b3951d491414dd9cc2f8d9597e7bc0a08a38044abb4492db3925a9cb0a: Status 404 returned error can't find the container with id 89cc86b3951d491414dd9cc2f8d9597e7bc0a08a38044abb4492db3925a9cb0a Jul 21 17:39:40.035730 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.035699 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:40.035793 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.035770 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:40.035835 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.035798 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:40.161761 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.161674 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" event={"ID":"ae5b927b-95ae-4533-b314-e697c7f010d3","Type":"ContainerStarted","Data":"f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5"} Jul 21 17:39:40.161761 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.161738 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" event={"ID":"ae5b927b-95ae-4533-b314-e697c7f010d3","Type":"ContainerStarted","Data":"89cc86b3951d491414dd9cc2f8d9597e7bc0a08a38044abb4492db3925a9cb0a"} Jul 21 17:39:40.185959 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.185664 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" podStartSLOduration=1.18564314 podStartE2EDuration="1.18564314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:39:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:39:40.182647969 +0000 UTC m=+1879.796192270" watchObservedRunningTime="2026-07-21 17:39:40.18564314 +0000 UTC m=+1879.799187442" Jul 21 17:39:40.895397 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:40.895362 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:41.905290 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:41.904997 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:42.170320 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:42.170168 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:42.171628 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:42.171608 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:43.995642 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:43.995621 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:39:44.104063 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.103971 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-envoy\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104063 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104007 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9bvc\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-kube-api-access-s9bvc\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104063 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104036 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-certs\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104063 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104051 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-istio-token\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104084 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-credential-socket\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104112 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00043531-98ba-46f0-a433-a8056a6675e8-istiod-ca-cert\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104133 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00043531-98ba-46f0-a433-a8056a6675e8-istio-podinfo\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104168 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-socket\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104209 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-data\") pod \"00043531-98ba-46f0-a433-a8056a6675e8\" (UID: \"00043531-98ba-46f0-a433-a8056a6675e8\") " Jul 21 17:39:44.104408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104326 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:44.104708 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104495 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:44.104708 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104640 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00043531-98ba-46f0-a433-a8056a6675e8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:39:44.104708 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104662 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.104708 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104681 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.104896 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.104725 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:44.105508 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.105483 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-data" (OuterVolumeSpecName: "istio-data") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:44.106286 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.106265 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:39:44.106601 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.106567 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-istio-token" (OuterVolumeSpecName: "istio-token") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:39:44.106761 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.106742 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/00043531-98ba-46f0-a433-a8056a6675e8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:39:44.106761 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.106751 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-kube-api-access-s9bvc" (OuterVolumeSpecName: "kube-api-access-s9bvc") pod "00043531-98ba-46f0-a433-a8056a6675e8" (UID: "00043531-98ba-46f0-a433-a8056a6675e8"). InnerVolumeSpecName "kube-api-access-s9bvc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:39:44.179664 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.179626 2565 generic.go:356] "Generic (PLEG): container finished" podID="00043531-98ba-46f0-a433-a8056a6675e8" containerID="ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b" exitCode=0 Jul 21 17:39:44.179840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.179674 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" Jul 21 17:39:44.179840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.179706 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" event={"ID":"00043531-98ba-46f0-a433-a8056a6675e8","Type":"ContainerDied","Data":"ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b"} Jul 21 17:39:44.179840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.179765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m" event={"ID":"00043531-98ba-46f0-a433-a8056a6675e8","Type":"ContainerDied","Data":"23b31efa96dbb1f83c25f57c04979c6c008498dc50c6933e290b55561e22d988"} Jul 21 17:39:44.179840 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.179791 2565 scope.go:117] "RemoveContainer" containerID="ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b" Jul 21 17:39:44.188251 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.188217 2565 scope.go:117] "RemoveContainer" containerID="ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184" Jul 21 17:39:44.195473 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.195454 2565 scope.go:117] "RemoveContainer" containerID="ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b" Jul 21 17:39:44.195743 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:39:44.195726 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b\": container with ID starting with ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b not found: ID does not exist" containerID="ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b" Jul 21 17:39:44.195791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.195756 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b"} err="failed to get container status \"ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b\": rpc error: code = NotFound desc = could not find container \"ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b\": container with ID starting with ac826384e1d64eaa1c7b1ccd1d2efe411a2ca699a44d4179fd8721236fb5115b not found: ID does not exist" Jul 21 17:39:44.195791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.195775 2565 scope.go:117] "RemoveContainer" containerID="ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184" Jul 21 17:39:44.196003 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:39:44.195990 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184\": container with ID starting with ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184 not found: ID does not exist" containerID="ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184" Jul 21 17:39:44.196041 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.196005 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184"} err="failed to get container status \"ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184\": rpc error: code = NotFound desc = could not find container \"ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184\": container with ID starting with ac54d3acac893e61d253e502bfdba76c8115527e5229c8091fe435a6122b0184 not found: ID does not exist" Jul 21 17:39:44.202509 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.202488 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m"] Jul 21 17:39:44.205910 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.205890 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.205910 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.205912 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9bvc\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-kube-api-access-s9bvc\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.206026 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.205922 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/00043531-98ba-46f0-a433-a8056a6675e8-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.206026 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.205931 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.206026 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.205938 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/00043531-98ba-46f0-a433-a8056a6675e8-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.206026 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.205947 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/00043531-98ba-46f0-a433-a8056a6675e8-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.206026 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.205958 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/00043531-98ba-46f0-a433-a8056a6675e8-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:39:44.208360 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:44.208342 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-07f072da-gw-openshift-default-6bf4bfb85f-vj75m"] Jul 21 17:39:45.032657 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:45.032625 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00043531-98ba-46f0-a433-a8056a6675e8" path="/var/lib/kubelet/pods/00043531-98ba-46f0-a433-a8056a6675e8/volumes" Jul 21 17:39:53.217168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:53.217127 2565 generic.go:356] "Generic (PLEG): container finished" podID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerID="f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5" exitCode=0 Jul 21 17:39:53.217606 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:53.217182 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" event={"ID":"ae5b927b-95ae-4533-b314-e697c7f010d3","Type":"ContainerDied","Data":"f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5"} Jul 21 17:39:53.217606 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:53.217552 2565 scope.go:117] "RemoveContainer" containerID="f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5" Jul 21 17:39:53.218531 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:53.218505 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:53.218604 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:53.218574 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:53.218641 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:53.218604 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:39:54.223462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:54.223425 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" event={"ID":"ae5b927b-95ae-4533-b314-e697c7f010d3","Type":"ContainerStarted","Data":"424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972"} Jul 21 17:39:54.895773 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:54.895737 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:54.896881 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:54.896860 2565 patch_prober.go:28] interesting pod/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 21 17:39:54.896985 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:54.896906 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 21 17:39:55.896290 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:55.896232 2565 patch_prober.go:28] interesting pod/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 21 17:39:55.896684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:55.896309 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 21 17:39:56.896432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:56.896393 2565 patch_prober.go:28] interesting pod/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 21 17:39:56.896828 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:56.896468 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 21 17:39:57.170817 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:57.170740 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:57.895958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:57.895926 2565 patch_prober.go:28] interesting pod/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Jul 21 17:39:57.896176 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:57.896005 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Jul 21 17:39:58.899432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:58.899405 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:39:58.900363 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:39:58.900346 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:40:23.665817 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:23.665781 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5"] Jul 21 17:40:23.666276 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:23.666033 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" containerID="cri-o://424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972" gracePeriod=30 Jul 21 17:40:24.438662 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.438615 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc"] Jul 21 17:40:24.439106 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.439082 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" Jul 21 17:40:24.439188 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.439108 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" Jul 21 17:40:24.439273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.439190 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" Jul 21 17:40:24.439273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.439203 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" Jul 21 17:40:24.442100 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.442076 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.444576 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.444556 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-336913ae-openshift-default-dockercfg-lvsvc\"" Jul 21 17:40:24.454587 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.454558 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc"] Jul 21 17:40:24.545882 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.545836 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-socket\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546067 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.545889 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-envoy\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546067 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.545931 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-token\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546067 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.545993 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb7a67-3242-48fb-b860-1cecf4b7b441-istiod-ca-cert\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546067 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.546023 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-credential-socket\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546067 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.546062 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nxmvz\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-kube-api-access-nxmvz\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.546109 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-certs\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.546135 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-podinfo\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.546357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.546181 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-data\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647142 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647113 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-certs\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647142 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647147 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-podinfo\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647430 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647168 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-data\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647430 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-socket\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647430 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647341 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-envoy\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647430 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647383 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-token\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647624 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb7a67-3242-48fb-b860-1cecf4b7b441-istiod-ca-cert\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647624 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647483 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-credential-socket\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647624 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647526 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nxmvz\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-kube-api-access-nxmvz\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647624 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647545 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-certs\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647858 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647831 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-socket\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.647858 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647844 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-data\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.648010 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.647960 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-credential-socket\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.648194 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.648173 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb7a67-3242-48fb-b860-1cecf4b7b441-istiod-ca-cert\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.649598 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.649577 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-envoy\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.649598 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.649587 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-podinfo\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.657139 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.657117 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-token\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.657528 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.657499 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nxmvz\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-kube-api-access-nxmvz\") pod \"e2e-derive-336913ae-openshift-default-7db754f799-tvrfc\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.752764 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.752733 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:24.884054 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.884027 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc"] Jul 21 17:40:24.886610 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:40:24.886583 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddceb7a67_3242_48fb_b860_1cecf4b7b441.slice/crio-251528405ec8e53c19c46cb4133f63287c94893dceaa1b2c32824db7a10aca60 WatchSource:0}: Error finding container 251528405ec8e53c19c46cb4133f63287c94893dceaa1b2c32824db7a10aca60: Status 404 returned error can't find the container with id 251528405ec8e53c19c46cb4133f63287c94893dceaa1b2c32824db7a10aca60 Jul 21 17:40:24.888723 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.888693 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:40:24.888783 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.888768 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:40:24.888825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:24.888795 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:40:25.347131 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:25.347100 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" event={"ID":"dceb7a67-3242-48fb-b860-1cecf4b7b441","Type":"ContainerStarted","Data":"0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1"} Jul 21 17:40:25.347334 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:25.347157 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" event={"ID":"dceb7a67-3242-48fb-b860-1cecf4b7b441","Type":"ContainerStarted","Data":"251528405ec8e53c19c46cb4133f63287c94893dceaa1b2c32824db7a10aca60"} Jul 21 17:40:25.368461 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:25.368413 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" podStartSLOduration=1.368397738 podStartE2EDuration="1.368397738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:40:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:40:25.366111069 +0000 UTC m=+1924.979655369" watchObservedRunningTime="2026-07-21 17:40:25.368397738 +0000 UTC m=+1924.981942039" Jul 21 17:40:25.753536 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:25.753508 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:25.758663 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:25.758636 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:26.351332 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:26.351296 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:26.352225 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:26.352205 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:28.914078 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.914057 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:40:28.986790 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986704 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tg99d\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-kube-api-access-tg99d\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.986790 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986737 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-token\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.986790 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986758 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-certs\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.987070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986807 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-credential-socket\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.987070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986852 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae5b927b-95ae-4533-b314-e697c7f010d3-istiod-ca-cert\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.987070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986897 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-envoy\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.987070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986931 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-data\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.987070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.986975 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-socket\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.987070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.987024 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-podinfo\") pod \"ae5b927b-95ae-4533-b314-e697c7f010d3\" (UID: \"ae5b927b-95ae-4533-b314-e697c7f010d3\") " Jul 21 17:40:28.987402 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.987103 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:40:28.987402 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.987172 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:40:28.987507 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.987398 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:28.987507 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.987446 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:28.987507 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.987399 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:40:28.987857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.987830 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae5b927b-95ae-4533-b314-e697c7f010d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:40:28.988349 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.988321 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:40:28.989074 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.989042 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:40:28.989347 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.989313 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:40:28.989436 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.989397 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:40:28.989436 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:28.989407 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-kube-api-access-tg99d" (OuterVolumeSpecName: "kube-api-access-tg99d") pod "ae5b927b-95ae-4533-b314-e697c7f010d3" (UID: "ae5b927b-95ae-4533-b314-e697c7f010d3"). InnerVolumeSpecName "kube-api-access-tg99d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:40:29.087882 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.087841 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tg99d\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-kube-api-access-tg99d\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:29.087882 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.087874 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:29.087882 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.087884 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ae5b927b-95ae-4533-b314-e697c7f010d3-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:29.087882 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.087893 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:29.088159 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.087901 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:29.088159 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.087908 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ae5b927b-95ae-4533-b314-e697c7f010d3-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:29.088159 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.087916 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ae5b927b-95ae-4533-b314-e697c7f010d3-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:40:29.364607 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.364576 2565 generic.go:356] "Generic (PLEG): container finished" podID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerID="424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972" exitCode=0 Jul 21 17:40:29.364781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.364621 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" event={"ID":"ae5b927b-95ae-4533-b314-e697c7f010d3","Type":"ContainerDied","Data":"424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972"} Jul 21 17:40:29.364781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.364622 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" Jul 21 17:40:29.364781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.364663 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" event={"ID":"ae5b927b-95ae-4533-b314-e697c7f010d3","Type":"ContainerDied","Data":"89cc86b3951d491414dd9cc2f8d9597e7bc0a08a38044abb4492db3925a9cb0a"} Jul 21 17:40:29.364781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.364684 2565 scope.go:117] "RemoveContainer" containerID="424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972" Jul 21 17:40:29.372867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.372851 2565 scope.go:117] "RemoveContainer" containerID="f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5" Jul 21 17:40:29.380640 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.380618 2565 scope.go:117] "RemoveContainer" containerID="424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972" Jul 21 17:40:29.380903 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:40:29.380881 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972\": container with ID starting with 424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972 not found: ID does not exist" containerID="424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972" Jul 21 17:40:29.380950 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.380914 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972"} err="failed to get container status \"424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972\": rpc error: code = NotFound desc = could not find container \"424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972\": container with ID starting with 424f9a3e14f3e03a34c4623e916f21c573c295844a010570f9800052c5236972 not found: ID does not exist" Jul 21 17:40:29.380950 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.380937 2565 scope.go:117] "RemoveContainer" containerID="f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5" Jul 21 17:40:29.381190 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:40:29.381171 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5\": container with ID starting with f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5 not found: ID does not exist" containerID="f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5" Jul 21 17:40:29.381264 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.381197 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5"} err="failed to get container status \"f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5\": rpc error: code = NotFound desc = could not find container \"f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5\": container with ID starting with f8a077a3e062aa7a4ec708f408de9e6a3bd0d325fd48f31a214f18824408b4d5 not found: ID does not exist" Jul 21 17:40:29.381303 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.381259 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5"] Jul 21 17:40:29.384596 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.384577 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5"] Jul 21 17:40:29.900974 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.900938 2565 patch_prober.go:28] interesting pod/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 21 17:40:29.901150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:29.900995 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-6b1d8ddb-openshift-default-f4c65df57-qlvb5" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 21 17:40:31.032179 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:31.032133 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" path="/var/lib/kubelet/pods/ae5b927b-95ae-4533-b314-e697c7f010d3/volumes" Jul 21 17:40:38.403020 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:38.402982 2565 generic.go:356] "Generic (PLEG): container finished" podID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerID="0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1" exitCode=0 Jul 21 17:40:38.403493 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:38.403055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" event={"ID":"dceb7a67-3242-48fb-b860-1cecf4b7b441","Type":"ContainerDied","Data":"0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1"} Jul 21 17:40:38.403493 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:38.403451 2565 scope.go:117] "RemoveContainer" containerID="0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1" Jul 21 17:40:38.404398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:38.404372 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:40:38.404457 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:38.404439 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:40:38.404502 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:38.404471 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:40:39.408943 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:39.408910 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" event={"ID":"dceb7a67-3242-48fb-b860-1cecf4b7b441","Type":"ContainerStarted","Data":"51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29"} Jul 21 17:40:39.754865 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:39.754824 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:39.755066 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:39.754958 2565 patch_prober.go:28] interesting pod/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 21 17:40:39.755066 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:39.755005 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 21 17:40:40.754383 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:40.753883 2565 patch_prober.go:28] interesting pod/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 21 17:40:40.754383 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:40.753948 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 21 17:40:41.352220 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:41.352169 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:41.753751 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:41.753707 2565 patch_prober.go:28] interesting pod/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 21 17:40:41.753932 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:41.753776 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 21 17:40:42.753382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:42.753336 2565 patch_prober.go:28] interesting pod/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Jul 21 17:40:42.753888 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:42.753406 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Jul 21 17:40:43.757278 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:43.757227 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:40:43.758307 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:40:43.758287 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:41:06.792158 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:06.792118 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc"] Jul 21 17:41:06.792838 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:06.792390 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" containerID="cri-o://51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29" gracePeriod=30 Jul 21 17:41:12.039125 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.039087 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:41:12.156830 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156742 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-token\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.156830 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156777 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-podinfo\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.156830 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156801 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-certs\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.156830 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156821 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb7a67-3242-48fb-b860-1cecf4b7b441-istiod-ca-cert\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.157168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156868 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-credential-socket\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.157168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156890 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-data\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.157168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156920 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nxmvz\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-kube-api-access-nxmvz\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.157168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.156964 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-socket\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.157168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.157000 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-envoy\") pod \"dceb7a67-3242-48fb-b860-1cecf4b7b441\" (UID: \"dceb7a67-3242-48fb-b860-1cecf4b7b441\") " Jul 21 17:41:12.157168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.157131 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:41:12.157168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.157149 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:41:12.157486 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.157275 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dceb7a67-3242-48fb-b860-1cecf4b7b441-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:41:12.157486 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.157290 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.157486 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.157309 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.157486 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.157425 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:41:12.158121 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.158086 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-data" (OuterVolumeSpecName: "istio-data") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:41:12.159208 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.159173 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:41:12.159208 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.159191 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:41:12.159409 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.159226 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-token" (OuterVolumeSpecName: "istio-token") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:41:12.159526 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.159509 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-kube-api-access-nxmvz" (OuterVolumeSpecName: "kube-api-access-nxmvz") pod "dceb7a67-3242-48fb-b860-1cecf4b7b441" (UID: "dceb7a67-3242-48fb-b860-1cecf4b7b441"). InnerVolumeSpecName "kube-api-access-nxmvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:41:12.257832 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.257792 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.257832 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.257830 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.257832 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.257839 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dceb7a67-3242-48fb-b860-1cecf4b7b441-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.257832 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.257849 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.258089 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.257857 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nxmvz\" (UniqueName: \"kubernetes.io/projected/dceb7a67-3242-48fb-b860-1cecf4b7b441-kube-api-access-nxmvz\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.258089 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.257867 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.258089 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.257875 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dceb7a67-3242-48fb-b860-1cecf4b7b441-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:41:12.544664 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.544628 2565 generic.go:356] "Generic (PLEG): container finished" podID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerID="51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29" exitCode=0 Jul 21 17:41:12.544860 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.544687 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" Jul 21 17:41:12.544860 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.544714 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" event={"ID":"dceb7a67-3242-48fb-b860-1cecf4b7b441","Type":"ContainerDied","Data":"51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29"} Jul 21 17:41:12.544860 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.544766 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc" event={"ID":"dceb7a67-3242-48fb-b860-1cecf4b7b441","Type":"ContainerDied","Data":"251528405ec8e53c19c46cb4133f63287c94893dceaa1b2c32824db7a10aca60"} Jul 21 17:41:12.544860 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.544788 2565 scope.go:117] "RemoveContainer" containerID="51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29" Jul 21 17:41:12.557563 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.557526 2565 scope.go:117] "RemoveContainer" containerID="0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1" Jul 21 17:41:12.565887 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.565861 2565 scope.go:117] "RemoveContainer" containerID="51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29" Jul 21 17:41:12.566146 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:41:12.566126 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29\": container with ID starting with 51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29 not found: ID does not exist" containerID="51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29" Jul 21 17:41:12.566225 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.566157 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29"} err="failed to get container status \"51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29\": rpc error: code = NotFound desc = could not find container \"51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29\": container with ID starting with 51694e297ffe1c2a4669057aa4d137366f5f962e5568922194afb5d09215ba29 not found: ID does not exist" Jul 21 17:41:12.566225 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.566183 2565 scope.go:117] "RemoveContainer" containerID="0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1" Jul 21 17:41:12.566484 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:41:12.566467 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1\": container with ID starting with 0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1 not found: ID does not exist" containerID="0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1" Jul 21 17:41:12.566568 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.566489 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1"} err="failed to get container status \"0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1\": rpc error: code = NotFound desc = could not find container \"0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1\": container with ID starting with 0beb1a8d754444dd80c4dfec412d97a09b282f5f9af7056f95b5e50c06061ef1 not found: ID does not exist" Jul 21 17:41:12.569086 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.569065 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc"] Jul 21 17:41:12.572369 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:12.572348 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-336913ae-openshift-default-7db754f799-tvrfc"] Jul 21 17:41:13.033751 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:41:13.033719 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" path="/var/lib/kubelet/pods/dceb7a67-3242-48fb-b860-1cecf4b7b441/volumes" Jul 21 17:42:49.487019 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.486939 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb"] Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487254 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487267 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487281 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487287 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487304 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487310 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487316 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487321 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="00043531-98ba-46f0-a433-a8056a6675e8" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487370 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487378 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" Jul 21 17:42:49.487477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.487386 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae5b927b-95ae-4533-b314-e697c7f010d3" containerName="istio-proxy" Jul 21 17:42:49.490224 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.490204 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.492698 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.492675 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-5151e12a-openshift-default-dockercfg-ms6kk\"" Jul 21 17:42:49.508978 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.508956 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb"] Jul 21 17:42:49.558223 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558189 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-certs\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558223 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558222 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-token\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558272 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dm72\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-kube-api-access-5dm72\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558303 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-envoy\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-podinfo\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558387 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c7561f-7630-4b65-bfb7-47484c13eef2-istiod-ca-cert\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-socket\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558422 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-credential-socket\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.558619 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.558458 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-data\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659165 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659131 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-credential-socket\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659180 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-data\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-certs\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659227 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-token\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659270 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dm72\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-kube-api-access-5dm72\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659312 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-envoy\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659338 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-podinfo\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c7561f-7630-4b65-bfb7-47484c13eef2-istiod-ca-cert\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-socket\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659793 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659681 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-certs\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659793 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659691 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-data\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659793 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659756 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-socket\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.659949 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.659908 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-credential-socket\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.660113 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.660092 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c7561f-7630-4b65-bfb7-47484c13eef2-istiod-ca-cert\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.661549 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.661529 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-envoy\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.661832 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.661813 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-podinfo\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.666798 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.666773 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dm72\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-kube-api-access-5dm72\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.666921 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.666903 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-token\") pod \"e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.799768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.799740 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:49.932011 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.931977 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb"] Jul 21 17:42:49.935000 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:42:49.934970 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0c7561f_7630_4b65_bfb7_47484c13eef2.slice/crio-74f780cd445938dcf4adaa16110a7f4ecbd5482ee11128116b54c726ad6bd5b1 WatchSource:0}: Error finding container 74f780cd445938dcf4adaa16110a7f4ecbd5482ee11128116b54c726ad6bd5b1: Status 404 returned error can't find the container with id 74f780cd445938dcf4adaa16110a7f4ecbd5482ee11128116b54c726ad6bd5b1 Jul 21 17:42:49.937094 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.937057 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:42:49.937177 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.937139 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:42:49.937215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:49.937178 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:42:50.943010 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:50.942973 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" event={"ID":"a0c7561f-7630-4b65-bfb7-47484c13eef2","Type":"ContainerStarted","Data":"df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89"} Jul 21 17:42:50.943436 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:50.943037 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" event={"ID":"a0c7561f-7630-4b65-bfb7-47484c13eef2","Type":"ContainerStarted","Data":"74f780cd445938dcf4adaa16110a7f4ecbd5482ee11128116b54c726ad6bd5b1"} Jul 21 17:42:50.962683 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:50.962634 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" podStartSLOduration=1.962618268 podStartE2EDuration="1.962618268s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:42:50.961258465 +0000 UTC m=+2070.574802759" watchObservedRunningTime="2026-07-21 17:42:50.962618268 +0000 UTC m=+2070.576162622" Jul 21 17:42:51.800301 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:51.800265 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:51.804768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:51.804745 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:51.947640 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:51.947609 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:42:51.948673 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:42:51.948654 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:43:06.949613 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:06.949578 2565 patch_prober.go:28] interesting pod/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 21 17:43:06.950035 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:06.949642 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 21 17:43:07.008758 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:07.008729 2565 generic.go:356] "Generic (PLEG): container finished" podID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerID="df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89" exitCode=0 Jul 21 17:43:07.008938 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:07.008802 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" event={"ID":"a0c7561f-7630-4b65-bfb7-47484c13eef2","Type":"ContainerDied","Data":"df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89"} Jul 21 17:43:07.009159 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:07.009145 2565 scope.go:117] "RemoveContainer" containerID="df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89" Jul 21 17:43:07.010061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:07.010036 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:43:07.010113 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:07.010102 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:43:07.010153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:07.010130 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:43:08.014697 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:08.014662 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" event={"ID":"a0c7561f-7630-4b65-bfb7-47484c13eef2","Type":"ContainerStarted","Data":"b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6"} Jul 21 17:43:08.800900 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:08.800853 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:43:08.801079 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:08.801036 2565 patch_prober.go:28] interesting pod/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 21 17:43:08.801124 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:08.801085 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 21 17:43:09.800172 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:09.800133 2565 patch_prober.go:28] interesting pod/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 21 17:43:09.800593 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:09.800201 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 21 17:43:10.800083 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:10.800049 2565 patch_prober.go:28] interesting pod/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" start-of-body= Jul 21 17:43:10.800273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:10.800106 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.28:15021/healthz/ready\": dial tcp 10.133.0.28:15021: connect: connection refused" Jul 21 17:43:11.848418 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:11.848384 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:43:11.848794 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:11.848650 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:43:11.849250 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:11.849217 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:43:26.677976 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:26.677935 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb"] Jul 21 17:43:26.678508 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:26.678293 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" containerID="cri-o://b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6" gracePeriod=30 Jul 21 17:43:26.851429 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:26.851388 2565 patch_prober.go:28] interesting pod/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 17:43:26.851617 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:26.851467 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 17:43:31.928066 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:31.928037 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:43:32.013699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013666 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-token\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.013699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013704 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dm72\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-kube-api-access-5dm72\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.013969 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013750 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-socket\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.013969 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013798 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c7561f-7630-4b65-bfb7-47484c13eef2-istiod-ca-cert\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.013969 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013829 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-data\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.013969 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013900 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-certs\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.013969 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013928 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-podinfo\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.013969 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013953 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-credential-socket\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.014445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.013980 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-envoy\") pod \"a0c7561f-7630-4b65-bfb7-47484c13eef2\" (UID: \"a0c7561f-7630-4b65-bfb7-47484c13eef2\") " Jul 21 17:43:32.014445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.014155 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:43:32.014445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.014185 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0c7561f-7630-4b65-bfb7-47484c13eef2-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:43:32.014445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.014204 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:43:32.014445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.014305 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.014445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.014326 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0c7561f-7630-4b65-bfb7-47484c13eef2-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.014445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.014381 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:43:32.014988 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.014961 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:43:32.016138 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.016115 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-kube-api-access-5dm72" (OuterVolumeSpecName: "kube-api-access-5dm72") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "kube-api-access-5dm72". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:43:32.016317 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.016297 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:43:32.016433 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.016416 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:43:32.016477 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.016419 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0c7561f-7630-4b65-bfb7-47484c13eef2" (UID: "a0c7561f-7630-4b65-bfb7-47484c13eef2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:43:32.114281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114227 2565 generic.go:356] "Generic (PLEG): container finished" podID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerID="b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6" exitCode=0 Jul 21 17:43:32.114281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114268 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" event={"ID":"a0c7561f-7630-4b65-bfb7-47484c13eef2","Type":"ContainerDied","Data":"b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6"} Jul 21 17:43:32.114510 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114289 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" Jul 21 17:43:32.114510 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114324 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb" event={"ID":"a0c7561f-7630-4b65-bfb7-47484c13eef2","Type":"ContainerDied","Data":"74f780cd445938dcf4adaa16110a7f4ecbd5482ee11128116b54c726ad6bd5b1"} Jul 21 17:43:32.114510 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114348 2565 scope.go:117] "RemoveContainer" containerID="b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6" Jul 21 17:43:32.114782 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114763 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.114903 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114787 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.114903 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114801 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.114903 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114813 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.114903 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114824 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.114903 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114835 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.114903 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.114844 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dm72\" (UniqueName: \"kubernetes.io/projected/a0c7561f-7630-4b65-bfb7-47484c13eef2-kube-api-access-5dm72\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:43:32.122777 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.122758 2565 scope.go:117] "RemoveContainer" containerID="df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89" Jul 21 17:43:32.129832 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.129817 2565 scope.go:117] "RemoveContainer" containerID="b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6" Jul 21 17:43:32.130071 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:43:32.130051 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6\": container with ID starting with b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6 not found: ID does not exist" containerID="b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6" Jul 21 17:43:32.130123 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.130078 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6"} err="failed to get container status \"b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6\": rpc error: code = NotFound desc = could not find container \"b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6\": container with ID starting with b6facc3b9722dbf63050da1cd3176bc1c56d9fb7060232f6512b5ca95bb224d6 not found: ID does not exist" Jul 21 17:43:32.130123 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.130095 2565 scope.go:117] "RemoveContainer" containerID="df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89" Jul 21 17:43:32.130345 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:43:32.130328 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89\": container with ID starting with df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89 not found: ID does not exist" containerID="df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89" Jul 21 17:43:32.130391 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.130351 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89"} err="failed to get container status \"df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89\": rpc error: code = NotFound desc = could not find container \"df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89\": container with ID starting with df9348a56318ab8694679e2a98b052c9fd4b22240b8933965e898a716b435f89 not found: ID does not exist" Jul 21 17:43:32.135613 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.135590 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb"] Jul 21 17:43:32.139019 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:32.138999 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-5151e12a-openshift-default-75c8f87555-vvqmb"] Jul 21 17:43:33.032539 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:33.032505 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" path="/var/lib/kubelet/pods/a0c7561f-7630-4b65-bfb7-47484c13eef2/volumes" Jul 21 17:43:46.829932 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.829894 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh"] Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830227 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830261 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830272 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830277 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830283 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830290 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830339 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" Jul 21 17:43:46.830462 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.830347 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="dceb7a67-3242-48fb-b860-1cecf4b7b441" containerName="istio-proxy" Jul 21 17:43:46.834213 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.834190 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.836620 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.836600 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f446106b-openshift-default-dockercfg-9qzdm\"" Jul 21 17:43:46.837071 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837043 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hnfz\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-kube-api-access-7hnfz\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837126 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837095 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-socket\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837161 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837147 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-credential-socket\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837223 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837202 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-envoy\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837285 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837256 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-podinfo\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837322 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837288 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-data\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837322 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837306 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-token\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837389 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837361 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d819d-d327-4107-a3f4-c2fbc99673ff-istiod-ca-cert\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.837422 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.837388 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-certs\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.842583 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.842558 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh"] Jul 21 17:43:46.938083 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938042 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-data\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938299 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938097 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-token\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938299 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938141 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d819d-d327-4107-a3f4-c2fbc99673ff-istiod-ca-cert\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938299 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938165 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-certs\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938299 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938220 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hnfz\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-kube-api-access-7hnfz\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938299 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938277 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-socket\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938573 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938341 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-credential-socket\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938573 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938386 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-envoy\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938573 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938414 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-podinfo\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.938962 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938938 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-credential-socket\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.939087 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938938 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-data\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.939087 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.938912 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-socket\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.939199 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.939180 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d819d-d327-4107-a3f4-c2fbc99673ff-istiod-ca-cert\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.939199 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.939190 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-certs\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.941062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.941039 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-envoy\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.941211 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.941192 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-podinfo\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.945545 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.945526 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-token\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:46.945665 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:46.945647 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hnfz\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-kube-api-access-7hnfz\") pod \"e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:47.145704 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:47.145613 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:47.283567 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:47.280492 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh"] Jul 21 17:43:47.285724 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:47.285676 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:43:47.285848 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:47.285748 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:43:47.285848 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:47.285777 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:43:48.179604 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:48.179565 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" event={"ID":"f58d819d-d327-4107-a3f4-c2fbc99673ff","Type":"ContainerStarted","Data":"3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9"} Jul 21 17:43:48.179992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:48.179629 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" event={"ID":"f58d819d-d327-4107-a3f4-c2fbc99673ff","Type":"ContainerStarted","Data":"97d12156d41ae03715d96abb41ed398c858d0af57303cd1dfd177478600319a1"} Jul 21 17:43:48.199651 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:48.199591 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podStartSLOduration=2.199569919 podStartE2EDuration="2.199569919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:43:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:43:48.198750003 +0000 UTC m=+2127.812294304" watchObservedRunningTime="2026-07-21 17:43:48.199569919 +0000 UTC m=+2127.813114222" Jul 21 17:43:49.146045 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:49.146004 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:49.150709 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:49.150686 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:49.183610 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:49.183587 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:43:49.184452 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:43:49.184435 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:44:05.109702 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.109607 2565 patch_prober.go:28] interesting pod/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": read tcp 10.133.0.2:34384->10.133.0.29:15021: read: connection reset by peer" start-of-body= Jul 21 17:44:05.109702 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.109674 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": read tcp 10.133.0.2:34384->10.133.0.29:15021: read: connection reset by peer" Jul 21 17:44:05.254933 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.254897 2565 generic.go:356] "Generic (PLEG): container finished" podID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerID="3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9" exitCode=0 Jul 21 17:44:05.255101 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.254971 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" event={"ID":"f58d819d-d327-4107-a3f4-c2fbc99673ff","Type":"ContainerDied","Data":"3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9"} Jul 21 17:44:05.255383 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.255364 2565 scope.go:117] "RemoveContainer" containerID="3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9" Jul 21 17:44:05.256352 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.256323 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:44:05.256449 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.256395 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:44:05.256449 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:05.256427 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:44:06.265645 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:06.265611 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" event={"ID":"f58d819d-d327-4107-a3f4-c2fbc99673ff","Type":"ContainerStarted","Data":"6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b"} Jul 21 17:44:07.146818 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:07.146783 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:44:07.147172 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:07.147148 2565 patch_prober.go:28] interesting pod/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 21 17:44:07.147366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:07.147337 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 21 17:44:08.147146 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:08.147103 2565 patch_prober.go:28] interesting pod/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 21 17:44:08.147786 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:08.147751 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 21 17:44:09.146576 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:09.146530 2565 patch_prober.go:28] interesting pod/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 21 17:44:09.146794 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:09.146615 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 21 17:44:10.146872 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:10.146833 2565 patch_prober.go:28] interesting pod/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" start-of-body= Jul 21 17:44:10.147479 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:10.147446 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.29:15021/healthz/ready\": dial tcp 10.133.0.29:15021: connect: connection refused" Jul 21 17:44:11.149699 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:11.149669 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:44:11.150125 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:11.149947 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:44:11.150662 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:44:11.150645 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:45:38.914958 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:38.914910 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh"] Jul 21 17:45:38.915518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:38.915136 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" containerID="cri-o://6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b" gracePeriod=30 Jul 21 17:45:41.152951 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:41.152916 2565 patch_prober.go:28] interesting pod/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 17:45:41.153357 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:41.152976 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 17:45:44.177107 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.177043 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:45:44.220043 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220017 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-certs\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220068 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-credential-socket\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220095 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7hnfz\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-kube-api-access-7hnfz\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220126 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-data\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220145 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-podinfo\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220179 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-socket\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220204 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d819d-d327-4107-a3f4-c2fbc99673ff-istiod-ca-cert\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220577 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220281 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-token\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220577 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220315 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-envoy\") pod \"f58d819d-d327-4107-a3f4-c2fbc99673ff\" (UID: \"f58d819d-d327-4107-a3f4-c2fbc99673ff\") " Jul 21 17:45:44.220577 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220319 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:45:44.220577 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.220552 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.221190 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.221150 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-data" (OuterVolumeSpecName: "istio-data") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:45:44.223004 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.222966 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:45:44.223130 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.223084 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f58d819d-d327-4107-a3f4-c2fbc99673ff-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:45:44.223375 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.223340 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:45:44.223620 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.223550 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:45:44.224986 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.224950 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-token" (OuterVolumeSpecName: "istio-token") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:45:44.224986 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.224967 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:45:44.225116 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.225014 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-kube-api-access-7hnfz" (OuterVolumeSpecName: "kube-api-access-7hnfz") pod "f58d819d-d327-4107-a3f4-c2fbc99673ff" (UID: "f58d819d-d327-4107-a3f4-c2fbc99673ff"). InnerVolumeSpecName "kube-api-access-7hnfz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:45:44.321930 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.321892 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.321930 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.321925 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f58d819d-d327-4107-a3f4-c2fbc99673ff-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.321930 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.321935 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.322265 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.321943 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.322265 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.321951 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.322265 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.321961 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7hnfz\" (UniqueName: \"kubernetes.io/projected/f58d819d-d327-4107-a3f4-c2fbc99673ff-kube-api-access-7hnfz\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.322265 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.321990 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.322265 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.322000 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f58d819d-d327-4107-a3f4-c2fbc99673ff-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:45:44.660601 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.660565 2565 generic.go:356] "Generic (PLEG): container finished" podID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerID="6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b" exitCode=0 Jul 21 17:45:44.660768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.660611 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" Jul 21 17:45:44.660768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.660614 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" event={"ID":"f58d819d-d327-4107-a3f4-c2fbc99673ff","Type":"ContainerDied","Data":"6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b"} Jul 21 17:45:44.660768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.660717 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh" event={"ID":"f58d819d-d327-4107-a3f4-c2fbc99673ff","Type":"ContainerDied","Data":"97d12156d41ae03715d96abb41ed398c858d0af57303cd1dfd177478600319a1"} Jul 21 17:45:44.660768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.660742 2565 scope.go:117] "RemoveContainer" containerID="6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b" Jul 21 17:45:44.669014 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.668993 2565 scope.go:117] "RemoveContainer" containerID="3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9" Jul 21 17:45:44.676291 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.676274 2565 scope.go:117] "RemoveContainer" containerID="6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b" Jul 21 17:45:44.676536 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:45:44.676515 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b\": container with ID starting with 6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b not found: ID does not exist" containerID="6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b" Jul 21 17:45:44.676585 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.676545 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b"} err="failed to get container status \"6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b\": rpc error: code = NotFound desc = could not find container \"6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b\": container with ID starting with 6b4ca0fbd71dd5927698d537decfde4a25836f718fbdda01fd0ff3f90f763c8b not found: ID does not exist" Jul 21 17:45:44.676585 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.676562 2565 scope.go:117] "RemoveContainer" containerID="3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9" Jul 21 17:45:44.676792 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:45:44.676776 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9\": container with ID starting with 3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9 not found: ID does not exist" containerID="3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9" Jul 21 17:45:44.676830 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.676798 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9"} err="failed to get container status \"3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9\": rpc error: code = NotFound desc = could not find container \"3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9\": container with ID starting with 3caeb5e006cc5ba55f7721d5c8d2ace81a3b9c6909b7830c5cc1b29c940161e9 not found: ID does not exist" Jul 21 17:45:44.681024 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.681003 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh"] Jul 21 17:45:44.684369 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:44.684350 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f446106b-openshift-default-6b495f7bbc-88cqh"] Jul 21 17:45:45.032680 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:45:45.032646 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" path="/var/lib/kubelet/pods/f58d819d-d327-4107-a3f4-c2fbc99673ff/volumes" Jul 21 17:46:35.089314 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089276 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h"] Jul 21 17:46:35.089742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089552 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" Jul 21 17:46:35.089742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089562 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" Jul 21 17:46:35.089742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089581 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" Jul 21 17:46:35.089742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089587 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" Jul 21 17:46:35.089742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089635 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" Jul 21 17:46:35.089742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089642 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58d819d-d327-4107-a3f4-c2fbc99673ff" containerName="istio-proxy" Jul 21 17:46:35.089742 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.089648 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0c7561f-7630-4b65-bfb7-47484c13eef2" containerName="istio-proxy" Jul 21 17:46:35.092478 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.092460 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.094830 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.094811 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-d1fcb9-openshift-default-dockercfg-dh2fp\"" Jul 21 17:46:35.103465 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.103438 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h"] Jul 21 17:46:35.231629 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231600 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-socket\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.231629 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231633 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-certs\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.231874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231653 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-data\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.231874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231669 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-credential-socket\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.231874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231732 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c858ff1-37f4-4983-aa61-325b180c54ec-istiod-ca-cert\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.231874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231774 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-token\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.231874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231848 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-podinfo\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.231874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-envoy\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.232116 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.231957 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rh7r\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-kube-api-access-5rh7r\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.332744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332712 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rh7r\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-kube-api-access-5rh7r\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.332943 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-socket\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.332943 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332793 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-certs\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.332943 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-data\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.332943 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332844 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-credential-socket\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.332943 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332881 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c858ff1-37f4-4983-aa61-325b180c54ec-istiod-ca-cert\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.332943 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332917 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-token\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.333273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-podinfo\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.333273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.332987 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-envoy\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.333379 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.333264 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-socket\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.333379 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.333347 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-data\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.333485 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.333400 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-credential-socket\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.333485 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.333449 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-certs\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.333677 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.333658 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c858ff1-37f4-4983-aa61-325b180c54ec-istiod-ca-cert\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.335341 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.335323 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-envoy\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.335463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.335445 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-podinfo\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.340161 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.340102 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-token\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.340161 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.340129 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rh7r\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-kube-api-access-5rh7r\") pod \"e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.403475 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.403441 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:35.548266 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.547956 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h"] Jul 21 17:46:35.551539 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.551507 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:35.551632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.551570 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:35.551632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.551596 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:35.862289 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.862177 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" event={"ID":"0c858ff1-37f4-4983-aa61-325b180c54ec","Type":"ContainerStarted","Data":"21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9"} Jul 21 17:46:35.862289 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.862257 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" event={"ID":"0c858ff1-37f4-4983-aa61-325b180c54ec","Type":"ContainerStarted","Data":"3b0c07bc282bb2fa4683ff571ff6ac1329afd1e334ff26e5dd21cae298a6eca5"} Jul 21 17:46:35.882410 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:35.882359 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" podStartSLOduration=0.882341786 podStartE2EDuration="882.341786ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:46:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:46:35.88038405 +0000 UTC m=+2295.493928350" watchObservedRunningTime="2026-07-21 17:46:35.882341786 +0000 UTC m=+2295.495886086" Jul 21 17:46:36.404131 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:36.404097 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:36.408968 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:36.408943 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:36.865782 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:36.865748 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:36.866618 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:36.866600 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:42.925309 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.925274 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l"] Jul 21 17:46:42.928690 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.928668 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.930935 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.930915 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-95fd83-openshift-default-dockercfg-4djdt\"" Jul 21 17:46:42.940392 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.940364 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l"] Jul 21 17:46:42.998596 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998534 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zj7jv\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-kube-api-access-zj7jv\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998596 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998584 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-envoy\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998596 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-token\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998617 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-credential-socket\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998661 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-data\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998700 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3325f8f3-e40b-401b-b4f8-2abaea741120-istiod-ca-cert\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998736 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-podinfo\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998763 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-socket\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:42.998867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:42.998787 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-certs\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099214 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-podinfo\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099214 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099215 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-socket\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099492 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099231 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-certs\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099492 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099302 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zj7jv\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-kube-api-access-zj7jv\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099492 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099334 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-envoy\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099492 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099358 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-token\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099492 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099383 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-credential-socket\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099492 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-data\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099492 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099440 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3325f8f3-e40b-401b-b4f8-2abaea741120-istiod-ca-cert\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099836 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099669 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-socket\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099836 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099767 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-certs\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099836 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099787 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-credential-socket\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.099960 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.099938 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-data\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.100175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.100153 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3325f8f3-e40b-401b-b4f8-2abaea741120-istiod-ca-cert\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.101691 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.101665 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-envoy\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.101841 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.101820 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-podinfo\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.107015 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.106987 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zj7jv\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-kube-api-access-zj7jv\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.107399 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.107372 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-token\") pod \"e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.239134 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.239042 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:43.376345 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.376307 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l"] Jul 21 17:46:43.380130 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:46:43.380103 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3325f8f3_e40b_401b_b4f8_2abaea741120.slice/crio-f6b6b521e2f1506bad1e7fc270e777776a0ef866540b1d0d71a6da3a5c5caf7b WatchSource:0}: Error finding container f6b6b521e2f1506bad1e7fc270e777776a0ef866540b1d0d71a6da3a5c5caf7b: Status 404 returned error can't find the container with id f6b6b521e2f1506bad1e7fc270e777776a0ef866540b1d0d71a6da3a5c5caf7b Jul 21 17:46:43.382303 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.382262 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:43.382376 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.382323 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:43.382376 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.382353 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:43.894105 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.894067 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" event={"ID":"3325f8f3-e40b-401b-b4f8-2abaea741120","Type":"ContainerStarted","Data":"14af753cf0a1e0afee6d6bb1272c24f8a0e8e02295c96c9da9cb931b898fb72a"} Jul 21 17:46:43.894316 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.894136 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" event={"ID":"3325f8f3-e40b-401b-b4f8-2abaea741120","Type":"ContainerStarted","Data":"f6b6b521e2f1506bad1e7fc270e777776a0ef866540b1d0d71a6da3a5c5caf7b"} Jul 21 17:46:43.912432 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:43.912379 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" podStartSLOduration=1.9123477119999999 podStartE2EDuration="1.912347712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:46:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:46:43.911283226 +0000 UTC m=+2303.524827526" watchObservedRunningTime="2026-07-21 17:46:43.912347712 +0000 UTC m=+2303.525892007" Jul 21 17:46:44.240098 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:44.240016 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:44.251594 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:44.251349 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:44.898301 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:44.898272 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:44.899340 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:44.899323 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:46:50.924771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:50.924739 2565 generic.go:356] "Generic (PLEG): container finished" podID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerID="21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9" exitCode=0 Jul 21 17:46:50.925175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:50.924783 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" event={"ID":"0c858ff1-37f4-4983-aa61-325b180c54ec","Type":"ContainerDied","Data":"21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9"} Jul 21 17:46:50.925175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:50.925167 2565 scope.go:117] "RemoveContainer" containerID="21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9" Jul 21 17:46:50.926106 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:50.926079 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:50.926165 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:50.926156 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:50.926216 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:50.926201 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:51.866358 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:51.866321 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:51.930470 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:51.930438 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" event={"ID":"0c858ff1-37f4-4983-aa61-325b180c54ec","Type":"ContainerStarted","Data":"0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91"} Jul 21 17:46:52.403847 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:52.403804 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:52.404963 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:52.404940 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 21 17:46:52.405086 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:52.404986 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 21 17:46:53.404178 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:53.404145 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 21 17:46:53.404951 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:53.404650 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 21 17:46:54.404126 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:54.404086 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 21 17:46:54.404349 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:54.404144 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 21 17:46:55.404490 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:55.404452 2565 patch_prober.go:28] interesting pod/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" start-of-body= Jul 21 17:46:55.404984 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:55.404520 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.30:15021/healthz/ready\": dial tcp 10.133.0.30:15021: connect: connection refused" Jul 21 17:46:56.408022 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:56.407988 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:56.408447 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:56.408269 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:56.408950 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:56.408931 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:46:59.899927 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.899893 2565 patch_prober.go:28] interesting pod/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 21 17:46:59.900382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.899960 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 21 17:46:59.989294 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.989187 2565 generic.go:356] "Generic (PLEG): container finished" podID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerID="14af753cf0a1e0afee6d6bb1272c24f8a0e8e02295c96c9da9cb931b898fb72a" exitCode=0 Jul 21 17:46:59.989294 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.989272 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" event={"ID":"3325f8f3-e40b-401b-b4f8-2abaea741120","Type":"ContainerDied","Data":"14af753cf0a1e0afee6d6bb1272c24f8a0e8e02295c96c9da9cb931b898fb72a"} Jul 21 17:46:59.989654 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.989640 2565 scope.go:117] "RemoveContainer" containerID="14af753cf0a1e0afee6d6bb1272c24f8a0e8e02295c96c9da9cb931b898fb72a" Jul 21 17:46:59.990707 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.990676 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:59.990786 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.990747 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:46:59.990786 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:46:59.990779 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:47:00.995000 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:00.994966 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" event={"ID":"3325f8f3-e40b-401b-b4f8-2abaea741120","Type":"ContainerStarted","Data":"eed5d46c9ea2f7c8a6d3f30aafb9a9eba64a5ad9a32af926ef29ffe2c7b5d950"} Jul 21 17:47:01.240058 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:01.240007 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:47:01.240294 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:01.240155 2565 patch_prober.go:28] interesting pod/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 21 17:47:01.240294 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:01.240203 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 21 17:47:02.239766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:02.239721 2565 patch_prober.go:28] interesting pod/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 21 17:47:02.240395 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:02.240358 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 21 17:47:03.240221 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:03.240182 2565 patch_prober.go:28] interesting pod/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 21 17:47:03.240843 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:03.240812 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 21 17:47:04.240344 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:04.240302 2565 patch_prober.go:28] interesting pod/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body= Jul 21 17:47:04.240965 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:04.240930 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" Jul 21 17:47:05.243141 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:05.243109 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:47:05.243559 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:05.243440 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:47:05.243993 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:05.243975 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:47:34.824487 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.824406 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77"] Jul 21 17:47:34.827629 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.827607 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:34.830865 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.830840 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-95fd83\"/\"test-model-95fd83-kserve-self-signed-certs\"" Jul 21 17:47:34.830988 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.830864 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-95fd83\"/\"default-dockercfg-6cp8b\"" Jul 21 17:47:34.830988 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.830880 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-95fd83\"/\"kube-root-ca.crt\"" Jul 21 17:47:34.830988 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.830847 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-95fd83\"/\"openshift-service-ca.crt\"" Jul 21 17:47:34.839567 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.839547 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77"] Jul 21 17:47:34.939298 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.939262 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-home\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:34.939298 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.939300 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-model-cache\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:34.939500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.939318 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0dfa986-476c-4892-9b16-796f87a782ea-tls-certs\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:34.939500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.939372 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcqfk\" (UniqueName: \"kubernetes.io/projected/b0dfa986-476c-4892-9b16-796f87a782ea-kube-api-access-bcqfk\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:34.939500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.939422 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-dshm\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:34.939500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:34.939460 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-tmp-dir\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.040504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.040473 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-model-cache\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.040504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.040508 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0dfa986-476c-4892-9b16-796f87a782ea-tls-certs\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.040700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.040534 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcqfk\" (UniqueName: \"kubernetes.io/projected/b0dfa986-476c-4892-9b16-796f87a782ea-kube-api-access-bcqfk\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.040700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.040575 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-dshm\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.040700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.040607 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-tmp-dir\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.040700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.040677 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-home\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.041055 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.041025 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-home\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.041179 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.041053 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-tmp-dir\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.041179 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.041080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-model-cache\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.042907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.042879 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-dshm\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.043004 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.042976 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0dfa986-476c-4892-9b16-796f87a782ea-tls-certs\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.048762 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.048732 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcqfk\" (UniqueName: \"kubernetes.io/projected/b0dfa986-476c-4892-9b16-796f87a782ea-kube-api-access-bcqfk\") pod \"test-model-95fd83-kserve-5b69645bcf-8rl77\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.137146 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.137066 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:35.262294 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:35.262264 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77"] Jul 21 17:47:35.264693 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:47:35.264659 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb0dfa986_476c_4892_9b16_796f87a782ea.slice/crio-4873e694a49307f0be9a8e56d0a6c5be99657376dfaf20e8a58dd5b4923be788 WatchSource:0}: Error finding container 4873e694a49307f0be9a8e56d0a6c5be99657376dfaf20e8a58dd5b4923be788: Status 404 returned error can't find the container with id 4873e694a49307f0be9a8e56d0a6c5be99657376dfaf20e8a58dd5b4923be788 Jul 21 17:47:36.135408 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:36.135368 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" event={"ID":"b0dfa986-476c-4892-9b16-796f87a782ea","Type":"ContainerStarted","Data":"aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a"} Jul 21 17:47:36.135792 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:36.135415 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" event={"ID":"b0dfa986-476c-4892-9b16-796f87a782ea","Type":"ContainerStarted","Data":"4873e694a49307f0be9a8e56d0a6c5be99657376dfaf20e8a58dd5b4923be788"} Jul 21 17:47:36.152654 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:36.152609 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" podStartSLOduration=2.152596026 podStartE2EDuration="2.152596026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:47:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:47:36.150685691 +0000 UTC m=+2355.764229993" watchObservedRunningTime="2026-07-21 17:47:36.152596026 +0000 UTC m=+2355.766140327" Jul 21 17:47:45.137956 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:45.137923 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:45.138582 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:45.138170 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:45.150472 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:45.150450 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:47:45.193379 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:47:45.193346 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:48:41.991731 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:41.991656 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77"] Jul 21 17:48:57.011610 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.011559 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" podUID="b0dfa986-476c-4892-9b16-796f87a782ea" containerName="main" containerID="cri-o://aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a" gracePeriod=45 Jul 21 17:48:57.251036 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.251010 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:48:57.357760 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.357680 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-tmp-dir\") pod \"b0dfa986-476c-4892-9b16-796f87a782ea\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " Jul 21 17:48:57.357760 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.357730 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-model-cache\") pod \"b0dfa986-476c-4892-9b16-796f87a782ea\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " Jul 21 17:48:57.357985 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.357764 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0dfa986-476c-4892-9b16-796f87a782ea-tls-certs\") pod \"b0dfa986-476c-4892-9b16-796f87a782ea\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " Jul 21 17:48:57.357985 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.357796 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-home\") pod \"b0dfa986-476c-4892-9b16-796f87a782ea\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " Jul 21 17:48:57.357985 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.357853 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcqfk\" (UniqueName: \"kubernetes.io/projected/b0dfa986-476c-4892-9b16-796f87a782ea-kube-api-access-bcqfk\") pod \"b0dfa986-476c-4892-9b16-796f87a782ea\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " Jul 21 17:48:57.357985 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.357876 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-dshm\") pod \"b0dfa986-476c-4892-9b16-796f87a782ea\" (UID: \"b0dfa986-476c-4892-9b16-796f87a782ea\") " Jul 21 17:48:57.357985 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.357918 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b0dfa986-476c-4892-9b16-796f87a782ea" (UID: "b0dfa986-476c-4892-9b16-796f87a782ea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:48:57.358184 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.358005 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-model-cache" (OuterVolumeSpecName: "model-cache") pod "b0dfa986-476c-4892-9b16-796f87a782ea" (UID: "b0dfa986-476c-4892-9b16-796f87a782ea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:48:57.358184 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.358107 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-home" (OuterVolumeSpecName: "home") pod "b0dfa986-476c-4892-9b16-796f87a782ea" (UID: "b0dfa986-476c-4892-9b16-796f87a782ea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:48:57.358281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.358203 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-tmp-dir\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:48:57.358281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.358219 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-model-cache\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:48:57.358281 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.358227 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-home\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:48:57.360024 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.359999 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0dfa986-476c-4892-9b16-796f87a782ea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b0dfa986-476c-4892-9b16-796f87a782ea" (UID: "b0dfa986-476c-4892-9b16-796f87a782ea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:48:57.360024 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.359997 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-dshm" (OuterVolumeSpecName: "dshm") pod "b0dfa986-476c-4892-9b16-796f87a782ea" (UID: "b0dfa986-476c-4892-9b16-796f87a782ea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:48:57.360175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.360041 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0dfa986-476c-4892-9b16-796f87a782ea-kube-api-access-bcqfk" (OuterVolumeSpecName: "kube-api-access-bcqfk") pod "b0dfa986-476c-4892-9b16-796f87a782ea" (UID: "b0dfa986-476c-4892-9b16-796f87a782ea"). InnerVolumeSpecName "kube-api-access-bcqfk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:48:57.459970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.459889 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcqfk\" (UniqueName: \"kubernetes.io/projected/b0dfa986-476c-4892-9b16-796f87a782ea-kube-api-access-bcqfk\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:48:57.459970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.459927 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b0dfa986-476c-4892-9b16-796f87a782ea-dshm\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:48:57.459970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.459942 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b0dfa986-476c-4892-9b16-796f87a782ea-tls-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:48:57.479041 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.479006 2565 generic.go:356] "Generic (PLEG): container finished" podID="b0dfa986-476c-4892-9b16-796f87a782ea" containerID="aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a" exitCode=0 Jul 21 17:48:57.479206 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.479049 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" Jul 21 17:48:57.479206 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.479091 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" event={"ID":"b0dfa986-476c-4892-9b16-796f87a782ea","Type":"ContainerDied","Data":"aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a"} Jul 21 17:48:57.479206 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.479141 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77" event={"ID":"b0dfa986-476c-4892-9b16-796f87a782ea","Type":"ContainerDied","Data":"4873e694a49307f0be9a8e56d0a6c5be99657376dfaf20e8a58dd5b4923be788"} Jul 21 17:48:57.479206 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.479163 2565 scope.go:117] "RemoveContainer" containerID="aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a" Jul 21 17:48:57.489388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.489362 2565 scope.go:117] "RemoveContainer" containerID="aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a" Jul 21 17:48:57.489704 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:48:57.489679 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a\": container with ID starting with aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a not found: ID does not exist" containerID="aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a" Jul 21 17:48:57.489774 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.489712 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a"} err="failed to get container status \"aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a\": rpc error: code = NotFound desc = could not find container \"aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a\": container with ID starting with aa0e3dceba373b40ec17681d0ee923de3a2c4e5c7f551e4c6cc82ab74b7a969a not found: ID does not exist" Jul 21 17:48:57.501220 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.501196 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77"] Jul 21 17:48:57.505143 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:57.505119 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-95fd83/test-model-95fd83-kserve-5b69645bcf-8rl77"] Jul 21 17:48:59.031686 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:48:59.031652 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0dfa986-476c-4892-9b16-796f87a782ea" path="/var/lib/kubelet/pods/b0dfa986-476c-4892-9b16-796f87a782ea/volumes" Jul 21 17:49:03.995624 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:03.995587 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h"] Jul 21 17:49:03.996037 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:03.995847 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" containerID="cri-o://0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91" gracePeriod=30 Jul 21 17:49:09.248682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.248658 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:49:09.264342 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264284 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-credential-socket\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264342 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264328 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-podinfo\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264375 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c858ff1-37f4-4983-aa61-325b180c54ec-istiod-ca-cert\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264399 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-token\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264421 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rh7r\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-kube-api-access-5rh7r\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264447 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-envoy\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264473 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-data\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264778 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264533 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:09.264778 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264548 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-socket\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264778 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264590 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-certs\") pod \"0c858ff1-37f4-4983-aa61-325b180c54ec\" (UID: \"0c858ff1-37f4-4983-aa61-325b180c54ec\") " Jul 21 17:49:09.264935 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264867 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:09.264935 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264875 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c858ff1-37f4-4983-aa61-325b180c54ec-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:49:09.264935 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.264921 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.265095 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.265073 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:09.265591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.265560 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-data" (OuterVolumeSpecName: "istio-data") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:09.266693 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.266658 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-kube-api-access-5rh7r" (OuterVolumeSpecName: "kube-api-access-5rh7r") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "kube-api-access-5rh7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:49:09.266795 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.266740 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:49:09.266795 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.266737 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:09.266918 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.266884 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-token" (OuterVolumeSpecName: "istio-token") pod "0c858ff1-37f4-4983-aa61-325b180c54ec" (UID: "0c858ff1-37f4-4983-aa61-325b180c54ec"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:49:09.365791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365758 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.365791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365785 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.365791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365794 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.365791 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365803 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0c858ff1-37f4-4983-aa61-325b180c54ec-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.366057 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365811 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.366057 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365820 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rh7r\" (UniqueName: \"kubernetes.io/projected/0c858ff1-37f4-4983-aa61-325b180c54ec-kube-api-access-5rh7r\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.366057 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365829 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.366057 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.365837 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0c858ff1-37f4-4983-aa61-325b180c54ec-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:09.532717 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.532633 2565 generic.go:356] "Generic (PLEG): container finished" podID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerID="0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91" exitCode=0 Jul 21 17:49:09.532717 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.532681 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" Jul 21 17:49:09.533140 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.532681 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" event={"ID":"0c858ff1-37f4-4983-aa61-325b180c54ec","Type":"ContainerDied","Data":"0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91"} Jul 21 17:49:09.533140 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.532814 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h" event={"ID":"0c858ff1-37f4-4983-aa61-325b180c54ec","Type":"ContainerDied","Data":"3b0c07bc282bb2fa4683ff571ff6ac1329afd1e334ff26e5dd21cae298a6eca5"} Jul 21 17:49:09.533140 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.532853 2565 scope.go:117] "RemoveContainer" containerID="0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91" Jul 21 17:49:09.541141 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.541123 2565 scope.go:117] "RemoveContainer" containerID="21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9" Jul 21 17:49:09.548212 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.548192 2565 scope.go:117] "RemoveContainer" containerID="0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91" Jul 21 17:49:09.548467 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:49:09.548449 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91\": container with ID starting with 0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91 not found: ID does not exist" containerID="0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91" Jul 21 17:49:09.548518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.548475 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91"} err="failed to get container status \"0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91\": rpc error: code = NotFound desc = could not find container \"0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91\": container with ID starting with 0a29a31ec7fa46a1b027452f69100f65044a6b45dd3819f1264f229836914f91 not found: ID does not exist" Jul 21 17:49:09.548518 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.548490 2565 scope.go:117] "RemoveContainer" containerID="21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9" Jul 21 17:49:09.548703 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:49:09.548687 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9\": container with ID starting with 21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9 not found: ID does not exist" containerID="21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9" Jul 21 17:49:09.548744 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.548707 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9"} err="failed to get container status \"21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9\": rpc error: code = NotFound desc = could not find container \"21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9\": container with ID starting with 21e5bb8c20f05d4adf0735bb38980994da68f3fa64637b4f670bc60cc227bcb9 not found: ID does not exist" Jul 21 17:49:09.553326 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.553304 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h"] Jul 21 17:49:09.559632 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:09.559613 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-d1fcb9-openshift-default-5b666cb5ff-dbm8h"] Jul 21 17:49:11.031581 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:11.031540 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" path="/var/lib/kubelet/pods/0c858ff1-37f4-4983-aa61-325b180c54ec/volumes" Jul 21 17:49:26.494181 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:26.494144 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l"] Jul 21 17:49:26.494592 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:26.494387 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" containerID="cri-o://eed5d46c9ea2f7c8a6d3f30aafb9a9eba64a5ad9a32af926ef29ffe2c7b5d950" gracePeriod=30 Jul 21 17:49:27.474421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474387 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p"] Jul 21 17:49:27.474678 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474666 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" Jul 21 17:49:27.474736 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474680 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" Jul 21 17:49:27.474736 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474696 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0dfa986-476c-4892-9b16-796f87a782ea" containerName="main" Jul 21 17:49:27.474736 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474702 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0dfa986-476c-4892-9b16-796f87a782ea" containerName="main" Jul 21 17:49:27.474826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474754 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" Jul 21 17:49:27.474826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474762 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" Jul 21 17:49:27.474826 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.474768 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0dfa986-476c-4892-9b16-796f87a782ea" containerName="main" Jul 21 17:49:27.477546 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.477526 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.479948 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.479925 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-54b201-openshift-default-dockercfg-ddclf\"" Jul 21 17:49:27.489413 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.489390 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p"] Jul 21 17:49:27.515416 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515384 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-credential-socket\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515416 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515418 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v2jzg\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-kube-api-access-v2jzg\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515443 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-certs\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515524 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-socket\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/011de000-2167-411f-a405-3cfa67022d71-istiod-ca-cert\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515575 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-data\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-envoy\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515614 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-istio-token\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.515825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.515693 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/011de000-2167-411f-a405-3cfa67022d71-istio-podinfo\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617027 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-socket\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617060 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/011de000-2167-411f-a405-3cfa67022d71-istiod-ca-cert\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617082 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-data\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617106 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-envoy\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617124 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-istio-token\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617175 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/011de000-2167-411f-a405-3cfa67022d71-istio-podinfo\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-credential-socket\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617223 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v2jzg\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-kube-api-access-v2jzg\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617607 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617409 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-certs\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617607 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617573 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-socket\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617607 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617599 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-data\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617768 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617635 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-credential-socket\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617881 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617861 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-certs\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.617881 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.617868 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/011de000-2167-411f-a405-3cfa67022d71-istiod-ca-cert\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.619481 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.619455 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-envoy\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.619612 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.619595 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/011de000-2167-411f-a405-3cfa67022d71-istio-podinfo\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.625061 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.625042 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-istio-token\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.625152 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.625133 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v2jzg\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-kube-api-access-v2jzg\") pod \"e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.787504 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.787470 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:27.921566 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.921534 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p"] Jul 21 17:49:27.925510 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:49:27.925483 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod011de000_2167_411f_a405_3cfa67022d71.slice/crio-473cd3935966ef7bee2b3eb087fff36d6f8cd7860b7a32e72e5b2c7536c4b88d WatchSource:0}: Error finding container 473cd3935966ef7bee2b3eb087fff36d6f8cd7860b7a32e72e5b2c7536c4b88d: Status 404 returned error can't find the container with id 473cd3935966ef7bee2b3eb087fff36d6f8cd7860b7a32e72e5b2c7536c4b88d Jul 21 17:49:27.927926 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.927897 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:27.927994 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.927963 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:27.928036 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:27.928008 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:28.612088 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:28.612054 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" event={"ID":"011de000-2167-411f-a405-3cfa67022d71","Type":"ContainerStarted","Data":"9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89"} Jul 21 17:49:28.612598 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:28.612109 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" event={"ID":"011de000-2167-411f-a405-3cfa67022d71","Type":"ContainerStarted","Data":"473cd3935966ef7bee2b3eb087fff36d6f8cd7860b7a32e72e5b2c7536c4b88d"} Jul 21 17:49:28.633168 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:28.633123 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" podStartSLOduration=1.633107616 podStartE2EDuration="1.633107616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:49:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:49:28.632141673 +0000 UTC m=+2468.245685975" watchObservedRunningTime="2026-07-21 17:49:28.633107616 +0000 UTC m=+2468.246651916" Jul 21 17:49:28.787993 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:28.787927 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:28.792916 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:28.792890 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:29.617310 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:29.617274 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:29.618280 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:29.618260 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:31.627714 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.627680 2565 generic.go:356] "Generic (PLEG): container finished" podID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerID="eed5d46c9ea2f7c8a6d3f30aafb9a9eba64a5ad9a32af926ef29ffe2c7b5d950" exitCode=0 Jul 21 17:49:31.628153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.627706 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" event={"ID":"3325f8f3-e40b-401b-b4f8-2abaea741120","Type":"ContainerDied","Data":"eed5d46c9ea2f7c8a6d3f30aafb9a9eba64a5ad9a32af926ef29ffe2c7b5d950"} Jul 21 17:49:31.628153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.627822 2565 scope.go:117] "RemoveContainer" containerID="14af753cf0a1e0afee6d6bb1272c24f8a0e8e02295c96c9da9cb931b898fb72a" Jul 21 17:49:31.746373 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.746348 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:49:31.852625 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852526 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-data\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.852625 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852581 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3325f8f3-e40b-401b-b4f8-2abaea741120-istiod-ca-cert\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.852625 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852612 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-socket\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.852907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852646 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-envoy\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.852907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852673 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-token\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.852907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852696 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-credential-socket\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.852907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852832 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zj7jv\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-kube-api-access-zj7jv\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.852907 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852895 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-certs\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.853163 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852933 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-podinfo\") pod \"3325f8f3-e40b-401b-b4f8-2abaea741120\" (UID: \"3325f8f3-e40b-401b-b4f8-2abaea741120\") " Jul 21 17:49:31.853163 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.853076 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:31.853163 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.852999 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3325f8f3-e40b-401b-b4f8-2abaea741120-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:49:31.853361 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.853285 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:31.853361 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.853291 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3325f8f3-e40b-401b-b4f8-2abaea741120-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.853361 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.853314 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.853361 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.853338 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:31.853579 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.853530 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-data" (OuterVolumeSpecName: "istio-data") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:31.855047 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.855020 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-token" (OuterVolumeSpecName: "istio-token") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:49:31.855159 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.855091 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:49:31.855327 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.855296 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-kube-api-access-zj7jv" (OuterVolumeSpecName: "kube-api-access-zj7jv") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "kube-api-access-zj7jv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:49:31.855327 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.855304 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3325f8f3-e40b-401b-b4f8-2abaea741120" (UID: "3325f8f3-e40b-401b-b4f8-2abaea741120"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:49:31.954070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.954032 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.954070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.954060 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.954070 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.954070 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zj7jv\" (UniqueName: \"kubernetes.io/projected/3325f8f3-e40b-401b-b4f8-2abaea741120-kube-api-access-zj7jv\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.954336 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.954082 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.954336 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.954098 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.954336 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.954105 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:31.954336 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:31.954114 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3325f8f3-e40b-401b-b4f8-2abaea741120-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:49:32.633152 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:32.633121 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" Jul 21 17:49:32.633648 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:32.633122 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l" event={"ID":"3325f8f3-e40b-401b-b4f8-2abaea741120","Type":"ContainerDied","Data":"f6b6b521e2f1506bad1e7fc270e777776a0ef866540b1d0d71a6da3a5c5caf7b"} Jul 21 17:49:32.633648 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:32.633282 2565 scope.go:117] "RemoveContainer" containerID="eed5d46c9ea2f7c8a6d3f30aafb9a9eba64a5ad9a32af926ef29ffe2c7b5d950" Jul 21 17:49:32.655048 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:32.655019 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l"] Jul 21 17:49:32.659839 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:32.659819 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-95fd83-openshift-default-d7bdd6d5c-c7g5l"] Jul 21 17:49:33.032125 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:33.032082 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" path="/var/lib/kubelet/pods/3325f8f3-e40b-401b-b4f8-2abaea741120/volumes" Jul 21 17:49:36.129185 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129155 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv"] Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129466 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129478 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129487 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129492 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129507 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129513 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c858ff1-37f4-4983-aa61-325b180c54ec" containerName="istio-proxy" Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129570 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" Jul 21 17:49:36.129674 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.129577 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3325f8f3-e40b-401b-b4f8-2abaea741120" containerName="istio-proxy" Jul 21 17:49:36.133900 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.133878 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.136336 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.136311 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-eb37d3-openshift-default-dockercfg-7bcsz\"" Jul 21 17:49:36.146819 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.146796 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv"] Jul 21 17:49:36.191825 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.191788 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccac1a7c-5c78-449d-a001-7c1e5528194e-istiod-ca-cert\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.191992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.191834 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-token\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.191992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.191858 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-socket\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.191992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.191910 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-credential-socket\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.191992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.191953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-envoy\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.192153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.192005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z4vq\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-kube-api-access-2z4vq\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.192153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.192035 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-certs\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.192153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.192049 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-data\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.192153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.192068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-podinfo\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.293497 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z4vq\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-kube-api-access-2z4vq\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.293712 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293522 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-certs\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.293712 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293546 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-data\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.293712 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293575 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-podinfo\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.293712 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293615 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccac1a7c-5c78-449d-a001-7c1e5528194e-istiod-ca-cert\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.293712 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293662 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-token\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.293712 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293695 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-socket\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.294069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293719 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-credential-socket\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.294069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.293761 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-envoy\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.294069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.294008 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-certs\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.294221 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.294069 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-socket\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.294221 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.294114 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-credential-socket\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.294365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.294347 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-data\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.294456 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.294440 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccac1a7c-5c78-449d-a001-7c1e5528194e-istiod-ca-cert\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.295992 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.295967 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-envoy\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.296176 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.296158 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-podinfo\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.301794 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.301763 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-token\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.302619 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.302596 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z4vq\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-kube-api-access-2z4vq\") pod \"e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.463567 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.463468 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:36.617111 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.617088 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv"] Jul 21 17:49:36.618882 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:49:36.618855 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccac1a7c_5c78_449d_a001_7c1e5528194e.slice/crio-1ed8eff6b5f439f39a93ed4ff9b5249ca4c3850fffabc119fa136fa33d849665 WatchSource:0}: Error finding container 1ed8eff6b5f439f39a93ed4ff9b5249ca4c3850fffabc119fa136fa33d849665: Status 404 returned error can't find the container with id 1ed8eff6b5f439f39a93ed4ff9b5249ca4c3850fffabc119fa136fa33d849665 Jul 21 17:49:36.620906 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.620876 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:36.620979 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.620937 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:36.620979 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.620966 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:36.663093 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:36.663060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" event={"ID":"ccac1a7c-5c78-449d-a001-7c1e5528194e","Type":"ContainerStarted","Data":"1ed8eff6b5f439f39a93ed4ff9b5249ca4c3850fffabc119fa136fa33d849665"} Jul 21 17:49:37.670142 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:37.670105 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" event={"ID":"ccac1a7c-5c78-449d-a001-7c1e5528194e","Type":"ContainerStarted","Data":"a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c"} Jul 21 17:49:37.690328 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:37.690270 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" podStartSLOduration=1.6902310109999998 podStartE2EDuration="1.690231011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:49:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:49:37.688286748 +0000 UTC m=+2477.301831049" watchObservedRunningTime="2026-07-21 17:49:37.690231011 +0000 UTC m=+2477.303775315" Jul 21 17:49:38.464361 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:38.464327 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:38.469530 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:38.469505 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:38.674948 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:38.674916 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:38.675874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:38.675854 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:41.688309 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:41.688278 2565 generic.go:356] "Generic (PLEG): container finished" podID="011de000-2167-411f-a405-3cfa67022d71" containerID="9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89" exitCode=0 Jul 21 17:49:41.688754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:41.688363 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" event={"ID":"011de000-2167-411f-a405-3cfa67022d71","Type":"ContainerDied","Data":"9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89"} Jul 21 17:49:41.688808 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:41.688797 2565 scope.go:117] "RemoveContainer" containerID="9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89" Jul 21 17:49:41.689753 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:41.689707 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:41.689853 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:41.689789 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:41.689853 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:41.689819 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:42.694676 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:42.694644 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" event={"ID":"011de000-2167-411f-a405-3cfa67022d71","Type":"ContainerStarted","Data":"b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f"} Jul 21 17:49:42.787891 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:42.787848 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:42.788968 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:42.788942 2565 patch_prober.go:28] interesting pod/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 21 17:49:42.789035 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:42.788998 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 21 17:49:43.788677 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:43.788645 2565 patch_prober.go:28] interesting pod/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 21 17:49:43.789063 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:43.788696 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 21 17:49:44.618215 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:44.618181 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:44.788798 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:44.788766 2565 patch_prober.go:28] interesting pod/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 21 17:49:44.789188 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:44.788814 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 21 17:49:45.788511 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:45.788470 2565 patch_prober.go:28] interesting pod/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" start-of-body= Jul 21 17:49:45.788716 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:45.788529 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.33:15021/healthz/ready\": dial tcp 10.133.0.33:15021: connect: connection refused" Jul 21 17:49:46.791589 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:46.791557 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:46.792495 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:46.792478 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:49:57.766434 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:57.766400 2565 generic.go:356] "Generic (PLEG): container finished" podID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerID="a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c" exitCode=0 Jul 21 17:49:57.766906 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:57.766451 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" event={"ID":"ccac1a7c-5c78-449d-a001-7c1e5528194e","Type":"ContainerDied","Data":"a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c"} Jul 21 17:49:57.766906 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:57.766794 2565 scope.go:117] "RemoveContainer" containerID="a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c" Jul 21 17:49:57.767775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:57.767751 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:57.767860 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:57.767847 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:57.767901 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:57.767887 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:49:58.773365 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:58.773326 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" event={"ID":"ccac1a7c-5c78-449d-a001-7c1e5528194e","Type":"ContainerStarted","Data":"a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e"} Jul 21 17:49:59.464635 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:59.464608 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:49:59.464833 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:59.464810 2565 patch_prober.go:28] interesting pod/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 21 17:49:59.464891 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:49:59.464871 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 21 17:50:00.465088 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:00.465043 2565 patch_prober.go:28] interesting pod/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 21 17:50:00.465524 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:00.465133 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 21 17:50:01.464507 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:01.464469 2565 patch_prober.go:28] interesting pod/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 21 17:50:01.464690 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:01.464526 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 21 17:50:02.464962 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:02.464923 2565 patch_prober.go:28] interesting pod/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" start-of-body= Jul 21 17:50:02.465390 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:02.464987 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.34:15021/healthz/ready\": dial tcp 10.133.0.34:15021: connect: connection refused" Jul 21 17:50:03.468600 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:03.468568 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:50:03.469066 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:03.468843 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:50:03.469611 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:03.469594 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:50:19.462093 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:19.462017 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p"] Jul 21 17:50:19.462659 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:19.462260 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" containerID="cri-o://b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f" gracePeriod=30 Jul 21 17:50:24.706978 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.706955 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:50:24.815057 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815024 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v2jzg\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-kube-api-access-v2jzg\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815057 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815060 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-istio-token\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815088 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/011de000-2167-411f-a405-3cfa67022d71-istio-podinfo\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815116 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-data\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815146 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-credential-socket\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815170 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-certs\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815277 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-envoy\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815315 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815304 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-socket\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815340 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/011de000-2167-411f-a405-3cfa67022d71-istiod-ca-cert\") pod \"011de000-2167-411f-a405-3cfa67022d71\" (UID: \"011de000-2167-411f-a405-3cfa67022d71\") " Jul 21 17:50:24.815766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815501 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:24.815766 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815665 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.815944 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815827 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:24.815944 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815866 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:24.815944 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.815904 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/011de000-2167-411f-a405-3cfa67022d71-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:50:24.816114 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.816018 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-data" (OuterVolumeSpecName: "istio-data") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:24.817472 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.817445 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/011de000-2167-411f-a405-3cfa67022d71-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:50:24.817585 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.817465 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-kube-api-access-v2jzg" (OuterVolumeSpecName: "kube-api-access-v2jzg") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "kube-api-access-v2jzg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:50:24.817659 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.817644 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-istio-token" (OuterVolumeSpecName: "istio-token") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:50:24.817812 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.817795 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "011de000-2167-411f-a405-3cfa67022d71" (UID: "011de000-2167-411f-a405-3cfa67022d71"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:24.882882 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.882848 2565 generic.go:356] "Generic (PLEG): container finished" podID="011de000-2167-411f-a405-3cfa67022d71" containerID="b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f" exitCode=0 Jul 21 17:50:24.883108 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.882894 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" event={"ID":"011de000-2167-411f-a405-3cfa67022d71","Type":"ContainerDied","Data":"b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f"} Jul 21 17:50:24.883108 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.882901 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" Jul 21 17:50:24.883108 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.882937 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p" event={"ID":"011de000-2167-411f-a405-3cfa67022d71","Type":"ContainerDied","Data":"473cd3935966ef7bee2b3eb087fff36d6f8cd7860b7a32e72e5b2c7536c4b88d"} Jul 21 17:50:24.883108 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.882958 2565 scope.go:117] "RemoveContainer" containerID="b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f" Jul 21 17:50:24.891322 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.891304 2565 scope.go:117] "RemoveContainer" containerID="9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89" Jul 21 17:50:24.898720 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.898702 2565 scope.go:117] "RemoveContainer" containerID="b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f" Jul 21 17:50:24.898956 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:50:24.898938 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f\": container with ID starting with b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f not found: ID does not exist" containerID="b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f" Jul 21 17:50:24.898995 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.898964 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f"} err="failed to get container status \"b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f\": rpc error: code = NotFound desc = could not find container \"b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f\": container with ID starting with b89f5877c81a259c39c0e4f2d0d43d527b8fab4f48fa297ffcd8d5e24a5ea10f not found: ID does not exist" Jul 21 17:50:24.898995 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.898992 2565 scope.go:117] "RemoveContainer" containerID="9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89" Jul 21 17:50:24.899207 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:50:24.899190 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89\": container with ID starting with 9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89 not found: ID does not exist" containerID="9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89" Jul 21 17:50:24.899269 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.899214 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89"} err="failed to get container status \"9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89\": rpc error: code = NotFound desc = could not find container \"9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89\": container with ID starting with 9194f127225b775a305f69700ccc5a189d04e6fd76e0d1756e5e902a515f0d89 not found: ID does not exist" Jul 21 17:50:24.903757 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.903736 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p"] Jul 21 17:50:24.907949 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.907929 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-54b201-openshift-default-59447dd7c9-k2t4p"] Jul 21 17:50:24.916598 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916581 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v2jzg\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-kube-api-access-v2jzg\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.916684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916600 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/011de000-2167-411f-a405-3cfa67022d71-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.916684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916611 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/011de000-2167-411f-a405-3cfa67022d71-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.916684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916620 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.916684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916629 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.916684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916637 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.916684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916644 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/011de000-2167-411f-a405-3cfa67022d71-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:24.916684 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:24.916652 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/011de000-2167-411f-a405-3cfa67022d71-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:25.032150 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:25.032115 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="011de000-2167-411f-a405-3cfa67022d71" path="/var/lib/kubelet/pods/011de000-2167-411f-a405-3cfa67022d71/volumes" Jul 21 17:50:46.920908 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:46.920873 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv"] Jul 21 17:50:46.921351 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:46.921117 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" containerID="cri-o://a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e" gracePeriod=30 Jul 21 17:50:47.902607 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.902575 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm"] Jul 21 17:50:47.902867 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.902855 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" Jul 21 17:50:47.902924 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.902868 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" Jul 21 17:50:47.902966 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.902932 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" Jul 21 17:50:47.902966 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.902939 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" Jul 21 17:50:47.905934 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.905914 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:47.909167 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.909149 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-43413b-openshift-default-dockercfg-dfb5x\"" Jul 21 17:50:47.917297 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:47.917276 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm"] Jul 21 17:50:48.013944 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.013911 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-envoy\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.013953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqcgf\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-kube-api-access-fqcgf\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.014009 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-certs\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.014052 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-data\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.014154 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istiod-ca-cert\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.014269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-socket\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.014298 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-token\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.014321 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-podinfo\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.014421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.014391 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-credential-socket\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115091 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115049 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-envoy\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115091 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115092 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqcgf\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-kube-api-access-fqcgf\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115122 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-certs\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115143 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-data\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115196 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istiod-ca-cert\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115295 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-socket\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115323 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-token\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115350 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-podinfo\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115398 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115388 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-credential-socket\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115769 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115592 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-data\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115769 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115628 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-certs\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115769 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-credential-socket\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.115874 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.115850 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-socket\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.116023 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.116001 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istiod-ca-cert\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.117484 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.117459 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-envoy\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.117903 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.117883 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-podinfo\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.124071 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.124052 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqcgf\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-kube-api-access-fqcgf\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.124205 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.124080 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-token\") pod \"e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.216422 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.216337 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:48.351269 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.351218 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm"] Jul 21 17:50:48.358069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.358033 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:50:48.358161 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.358115 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:50:48.358161 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.358154 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:50:48.471754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.471654 2565 patch_prober.go:28] interesting pod/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 21 17:50:48.471754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.471724 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 21 17:50:48.987654 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.987623 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" event={"ID":"af0f3057-ebeb-4a36-9ce9-2617b08b180c","Type":"ContainerStarted","Data":"10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0"} Jul 21 17:50:48.987837 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:48.987673 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" event={"ID":"af0f3057-ebeb-4a36-9ce9-2617b08b180c","Type":"ContainerStarted","Data":"be1d0e900a4052e9a76a8eda7a631383900f1aa7bab6234c6e24e36193abbf88"} Jul 21 17:50:49.015082 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:49.015024 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" podStartSLOduration=2.015003995 podStartE2EDuration="2.015003995s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:50:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:50:49.013657199 +0000 UTC m=+2548.627201499" watchObservedRunningTime="2026-07-21 17:50:49.015003995 +0000 UTC m=+2548.628548297" Jul 21 17:50:49.216981 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:49.216948 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:49.218289 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:49.218265 2565 patch_prober.go:28] interesting pod/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 21 17:50:49.218421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:49.218311 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 21 17:50:50.220811 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:50.220786 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:50.996754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:50.996726 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:50.997726 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:50.997709 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:50:52.166268 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.166228 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:50:52.349165 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349130 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-token\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349165 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349171 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-certs\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349450 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349210 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z4vq\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-kube-api-access-2z4vq\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349450 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349275 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccac1a7c-5c78-449d-a001-7c1e5528194e-istiod-ca-cert\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349450 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349337 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-socket\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349450 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349375 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-credential-socket\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349450 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349397 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-data\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349450 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349421 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-envoy\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349450 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349443 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-podinfo\") pod \"ccac1a7c-5c78-449d-a001-7c1e5528194e\" (UID: \"ccac1a7c-5c78-449d-a001-7c1e5528194e\") " Jul 21 17:50:52.349802 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349568 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:52.349802 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349636 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:52.349802 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349720 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccac1a7c-5c78-449d-a001-7c1e5528194e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:50:52.349802 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349750 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.349802 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349770 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.350072 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.349988 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:52.350367 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.350339 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-data" (OuterVolumeSpecName: "istio-data") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:52.351486 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.351463 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-token" (OuterVolumeSpecName: "istio-token") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:50:52.351587 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.351543 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:50:52.351587 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.351547 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-kube-api-access-2z4vq" (OuterVolumeSpecName: "kube-api-access-2z4vq") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "kube-api-access-2z4vq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:50:52.351668 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.351625 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ccac1a7c-5c78-449d-a001-7c1e5528194e" (UID: "ccac1a7c-5c78-449d-a001-7c1e5528194e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:50:52.450543 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.450510 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.450543 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.450536 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.450543 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.450546 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.450759 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.450553 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.450759 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.450561 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.450759 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.450571 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z4vq\" (UniqueName: \"kubernetes.io/projected/ccac1a7c-5c78-449d-a001-7c1e5528194e-kube-api-access-2z4vq\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:52.450759 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:52.450580 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccac1a7c-5c78-449d-a001-7c1e5528194e-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:50:53.006071 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.006044 2565 generic.go:356] "Generic (PLEG): container finished" podID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerID="a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e" exitCode=0 Jul 21 17:50:53.006286 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.006097 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" Jul 21 17:50:53.006286 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.006134 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" event={"ID":"ccac1a7c-5c78-449d-a001-7c1e5528194e","Type":"ContainerDied","Data":"a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e"} Jul 21 17:50:53.006286 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.006203 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv" event={"ID":"ccac1a7c-5c78-449d-a001-7c1e5528194e","Type":"ContainerDied","Data":"1ed8eff6b5f439f39a93ed4ff9b5249ca4c3850fffabc119fa136fa33d849665"} Jul 21 17:50:53.006286 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.006230 2565 scope.go:117] "RemoveContainer" containerID="a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e" Jul 21 17:50:53.014829 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.014816 2565 scope.go:117] "RemoveContainer" containerID="a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c" Jul 21 17:50:53.022266 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.022224 2565 scope.go:117] "RemoveContainer" containerID="a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e" Jul 21 17:50:53.022535 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:50:53.022516 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e\": container with ID starting with a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e not found: ID does not exist" containerID="a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e" Jul 21 17:50:53.022619 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.022543 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e"} err="failed to get container status \"a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e\": rpc error: code = NotFound desc = could not find container \"a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e\": container with ID starting with a2a4dcf5c88c94f19a1343065856ccbd9d473f6e90cf8387ce6a9a482dd3310e not found: ID does not exist" Jul 21 17:50:53.022619 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.022568 2565 scope.go:117] "RemoveContainer" containerID="a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c" Jul 21 17:50:53.022795 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:50:53.022782 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c\": container with ID starting with a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c not found: ID does not exist" containerID="a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c" Jul 21 17:50:53.022834 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.022797 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c"} err="failed to get container status \"a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c\": rpc error: code = NotFound desc = could not find container \"a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c\": container with ID starting with a0d8cc4476dd7d399a154934f3bc42a2e8f97059cc97ee40df0ed57602b9e07c not found: ID does not exist" Jul 21 17:50:53.032352 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.032330 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv"] Jul 21 17:50:53.041555 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:53.041534 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-eb37d3-openshift-default-5f9bbcd8b5-z72jv"] Jul 21 17:50:55.031568 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:55.031537 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" path="/var/lib/kubelet/pods/ccac1a7c-5c78-449d-a001-7c1e5528194e/volumes" Jul 21 17:50:56.179333 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179302 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm"] Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179599 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179609 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="011de000-2167-411f-a405-3cfa67022d71" containerName="istio-proxy" Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179618 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179624 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179633 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179639 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179696 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" Jul 21 17:50:56.179700 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.179702 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccac1a7c-5c78-449d-a001-7c1e5528194e" containerName="istio-proxy" Jul 21 17:50:56.183865 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.183846 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.186358 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.186339 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-046b27-openshift-default-dockercfg-rjvd4\"" Jul 21 17:50:56.198544 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.198522 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm"] Jul 21 17:50:56.283043 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283010 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-podinfo\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283043 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283048 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-certs\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283080 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-credential-socket\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283142 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-socket\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283172 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clbgz\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-kube-api-access-clbgz\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283197 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-token\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283227 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-envoy\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283269 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-data\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.283382 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.283296 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istiod-ca-cert\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.383896 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.383855 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-socket\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384095 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.383907 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-clbgz\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-kube-api-access-clbgz\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384095 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.383941 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-token\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384095 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.383990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-envoy\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384095 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-data\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384095 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istiod-ca-cert\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384095 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384085 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-podinfo\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384470 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384215 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-certs\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384470 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384308 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-socket\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384470 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384327 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-credential-socket\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384648 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384625 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-certs\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384713 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384652 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-data\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.384713 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.384681 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-credential-socket\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.385125 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.385101 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istiod-ca-cert\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.386309 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.386288 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-envoy\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.386531 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.386515 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-podinfo\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.392912 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.392892 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-token\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.393153 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.393136 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-clbgz\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-kube-api-access-clbgz\") pod \"e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.493865 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.493830 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:56.625887 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.625864 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm"] Jul 21 17:50:56.628318 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:50:56.628292 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb950a3b0_9b89_4d66_8d08_ab78a4477efe.slice/crio-97622ec1ed802f9ad695953fd1f5f46b07cae228d7965d1df8b9a7876da348bd WatchSource:0}: Error finding container 97622ec1ed802f9ad695953fd1f5f46b07cae228d7965d1df8b9a7876da348bd: Status 404 returned error can't find the container with id 97622ec1ed802f9ad695953fd1f5f46b07cae228d7965d1df8b9a7876da348bd Jul 21 17:50:56.630458 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.630427 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:50:56.630545 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.630496 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:50:56.630545 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:56.630531 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:50:57.024103 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:57.024059 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" event={"ID":"b950a3b0-9b89-4d66-8d08-ab78a4477efe","Type":"ContainerStarted","Data":"650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487"} Jul 21 17:50:57.024310 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:57.024165 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" event={"ID":"b950a3b0-9b89-4d66-8d08-ab78a4477efe","Type":"ContainerStarted","Data":"97622ec1ed802f9ad695953fd1f5f46b07cae228d7965d1df8b9a7876da348bd"} Jul 21 17:50:57.049391 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:57.049343 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" podStartSLOduration=1.049328767 podStartE2EDuration="1.049328767s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:50:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:50:57.047869723 +0000 UTC m=+2556.661414023" watchObservedRunningTime="2026-07-21 17:50:57.049328767 +0000 UTC m=+2556.662873067" Jul 21 17:50:57.494346 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:57.494317 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:57.498625 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:57.498598 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:58.028463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:58.028435 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:50:58.029336 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:50:58.029316 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:51:05.064289 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:05.064256 2565 generic.go:356] "Generic (PLEG): container finished" podID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerID="10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0" exitCode=0 Jul 21 17:51:05.064289 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:05.064267 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" event={"ID":"af0f3057-ebeb-4a36-9ce9-2617b08b180c","Type":"ContainerDied","Data":"10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0"} Jul 21 17:51:05.064759 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:05.064666 2565 scope.go:117] "RemoveContainer" containerID="10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0" Jul 21 17:51:05.065666 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:05.065641 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:51:05.065732 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:05.065708 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:51:05.065778 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:05.065736 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:51:05.997703 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:05.997656 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:51:06.074334 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:06.074291 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" event={"ID":"af0f3057-ebeb-4a36-9ce9-2617b08b180c","Type":"ContainerStarted","Data":"a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863"} Jul 21 17:51:06.217062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:06.217028 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:51:06.218556 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:06.218526 2565 patch_prober.go:28] interesting pod/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 21 17:51:06.218679 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:06.218571 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 21 17:51:07.216807 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:07.216760 2565 patch_prober.go:28] interesting pod/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 21 17:51:07.217332 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:07.216829 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 21 17:51:08.216815 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:08.216771 2565 patch_prober.go:28] interesting pod/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 21 17:51:08.217226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:08.216835 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 21 17:51:09.217343 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:09.217295 2565 patch_prober.go:28] interesting pod/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" start-of-body= Jul 21 17:51:09.217765 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:09.217379 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.35:15021/healthz/ready\": dial tcp 10.133.0.35:15021: connect: connection refused" Jul 21 17:51:10.220568 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:10.220539 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:51:10.221015 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:10.220831 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:51:10.221663 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:10.221646 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:51:11.106142 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:11.106112 2565 generic.go:356] "Generic (PLEG): container finished" podID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerID="650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487" exitCode=0 Jul 21 17:51:11.106376 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:11.106182 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" event={"ID":"b950a3b0-9b89-4d66-8d08-ab78a4477efe","Type":"ContainerDied","Data":"650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487"} Jul 21 17:51:11.106645 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:11.106628 2565 scope.go:117] "RemoveContainer" containerID="650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487" Jul 21 17:51:11.107622 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:11.107599 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:51:11.107675 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:11.107663 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:51:11.107709 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:11.107693 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236228Ki","pods":"250"} Jul 21 17:51:12.112909 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:12.112876 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" event={"ID":"b950a3b0-9b89-4d66-8d08-ab78a4477efe","Type":"ContainerStarted","Data":"7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903"} Jul 21 17:51:12.495064 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:12.495026 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:51:12.496183 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:12.496161 2565 patch_prober.go:28] interesting pod/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 21 17:51:12.496257 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:12.496212 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 21 17:51:13.032378 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:13.032350 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:51:13.494602 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:13.494571 2565 patch_prober.go:28] interesting pod/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 21 17:51:13.494989 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:13.494642 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 21 17:51:14.494221 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:14.494175 2565 patch_prober.go:28] interesting pod/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 21 17:51:14.494421 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:14.494265 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 21 17:51:15.495050 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:15.495016 2565 patch_prober.go:28] interesting pod/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" start-of-body= Jul 21 17:51:15.495444 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:15.495081 2565 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.36:15021/healthz/ready\": dial tcp 10.133.0.36:15021: connect: connection refused" Jul 21 17:51:16.498069 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:16.498038 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:51:16.499122 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:16.499102 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:51:50.235993 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.235919 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2"] Jul 21 17:51:50.239199 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.239181 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.242612 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.242587 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"default-dockercfg-89sxf\"" Jul 21 17:51:50.242732 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.242666 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"auth-test-model-046b27-kserve-self-signed-certs\"" Jul 21 17:51:50.242732 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.242718 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"openshift-service-ca.crt\"" Jul 21 17:51:50.243041 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.243025 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"kube-root-ca.crt\"" Jul 21 17:51:50.248306 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.248285 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2"] Jul 21 17:51:50.256977 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.256957 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-dshm\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.257060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.257034 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36f5c03f-d215-4609-a5cc-b1145f133bea-tls-certs\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.257060 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.257054 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-home\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.257138 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.257074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-model-cache\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.257138 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.257089 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46rv4\" (UniqueName: \"kubernetes.io/projected/36f5c03f-d215-4609-a5cc-b1145f133bea-kube-api-access-46rv4\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.257138 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.257126 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-tmp-dir\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.357506 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357470 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36f5c03f-d215-4609-a5cc-b1145f133bea-tls-certs\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.357506 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357507 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-home\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.357764 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357527 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-model-cache\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.357764 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357548 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46rv4\" (UniqueName: \"kubernetes.io/projected/36f5c03f-d215-4609-a5cc-b1145f133bea-kube-api-access-46rv4\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.357764 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357568 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-tmp-dir\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.357764 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357678 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-dshm\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.357953 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357909 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-home\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.358005 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357982 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-model-cache\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.358005 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.357991 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-tmp-dir\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.359946 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.359915 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-dshm\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.360086 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.360068 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36f5c03f-d215-4609-a5cc-b1145f133bea-tls-certs\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.365413 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.365386 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46rv4\" (UniqueName: \"kubernetes.io/projected/36f5c03f-d215-4609-a5cc-b1145f133bea-kube-api-access-46rv4\") pod \"auth-test-model-046b27-kserve-8fff85f5-rwdc2\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.548782 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.548752 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:51:50.678067 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:50.678039 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2"] Jul 21 17:51:50.679979 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:51:50.679947 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod36f5c03f_d215_4609_a5cc_b1145f133bea.slice/crio-ac1a63c6cf857acdfb43e5e63b1fbb4be6edaa086be36b0bafc11088052c8943 WatchSource:0}: Error finding container ac1a63c6cf857acdfb43e5e63b1fbb4be6edaa086be36b0bafc11088052c8943: Status 404 returned error can't find the container with id ac1a63c6cf857acdfb43e5e63b1fbb4be6edaa086be36b0bafc11088052c8943 Jul 21 17:51:51.278445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:51.278411 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" event={"ID":"36f5c03f-d215-4609-a5cc-b1145f133bea","Type":"ContainerStarted","Data":"47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068"} Jul 21 17:51:51.278445 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:51.278456 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" event={"ID":"36f5c03f-d215-4609-a5cc-b1145f133bea","Type":"ContainerStarted","Data":"ac1a63c6cf857acdfb43e5e63b1fbb4be6edaa086be36b0bafc11088052c8943"} Jul 21 17:51:51.302635 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:51:51.302589 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" podStartSLOduration=1.302574657 podStartE2EDuration="1.302574657s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:51:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:51:51.301483712 +0000 UTC m=+2610.915028016" watchObservedRunningTime="2026-07-21 17:51:51.302574657 +0000 UTC m=+2610.916118957" Jul 21 17:52:00.549093 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:52:00.549050 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:52:00.549093 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:52:00.549103 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:52:00.561460 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:52:00.561437 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:52:01.331898 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:52:01.331871 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:53:55.936127 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:53:55.936093 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2"] Jul 21 17:54:10.956133 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:10.956072 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" podUID="36f5c03f-d215-4609-a5cc-b1145f133bea" containerName="main" containerID="cri-o://47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068" gracePeriod=45 Jul 21 17:54:11.194483 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.194456 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:54:11.338385 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338352 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-home\") pod \"36f5c03f-d215-4609-a5cc-b1145f133bea\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " Jul 21 17:54:11.338574 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338397 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-46rv4\" (UniqueName: \"kubernetes.io/projected/36f5c03f-d215-4609-a5cc-b1145f133bea-kube-api-access-46rv4\") pod \"36f5c03f-d215-4609-a5cc-b1145f133bea\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " Jul 21 17:54:11.338574 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338446 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36f5c03f-d215-4609-a5cc-b1145f133bea-tls-certs\") pod \"36f5c03f-d215-4609-a5cc-b1145f133bea\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " Jul 21 17:54:11.338574 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338469 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-dshm\") pod \"36f5c03f-d215-4609-a5cc-b1145f133bea\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " Jul 21 17:54:11.338574 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338517 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-tmp-dir\") pod \"36f5c03f-d215-4609-a5cc-b1145f133bea\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " Jul 21 17:54:11.338574 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338574 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-model-cache\") pod \"36f5c03f-d215-4609-a5cc-b1145f133bea\" (UID: \"36f5c03f-d215-4609-a5cc-b1145f133bea\") " Jul 21 17:54:11.338822 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338686 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-home" (OuterVolumeSpecName: "home") pod "36f5c03f-d215-4609-a5cc-b1145f133bea" (UID: "36f5c03f-d215-4609-a5cc-b1145f133bea"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:54:11.338885 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338825 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-home\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:54:11.338885 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338861 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "36f5c03f-d215-4609-a5cc-b1145f133bea" (UID: "36f5c03f-d215-4609-a5cc-b1145f133bea"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:54:11.338984 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.338911 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-model-cache" (OuterVolumeSpecName: "model-cache") pod "36f5c03f-d215-4609-a5cc-b1145f133bea" (UID: "36f5c03f-d215-4609-a5cc-b1145f133bea"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:54:11.340675 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.340654 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-dshm" (OuterVolumeSpecName: "dshm") pod "36f5c03f-d215-4609-a5cc-b1145f133bea" (UID: "36f5c03f-d215-4609-a5cc-b1145f133bea"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:54:11.340738 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.340694 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36f5c03f-d215-4609-a5cc-b1145f133bea-kube-api-access-46rv4" (OuterVolumeSpecName: "kube-api-access-46rv4") pod "36f5c03f-d215-4609-a5cc-b1145f133bea" (UID: "36f5c03f-d215-4609-a5cc-b1145f133bea"). InnerVolumeSpecName "kube-api-access-46rv4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:54:11.340781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.340757 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/36f5c03f-d215-4609-a5cc-b1145f133bea-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "36f5c03f-d215-4609-a5cc-b1145f133bea" (UID: "36f5c03f-d215-4609-a5cc-b1145f133bea"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:54:11.440182 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.440132 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-tmp-dir\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:54:11.440182 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.440175 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-model-cache\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:54:11.440182 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.440184 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-46rv4\" (UniqueName: \"kubernetes.io/projected/36f5c03f-d215-4609-a5cc-b1145f133bea-kube-api-access-46rv4\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:54:11.440182 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.440195 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/36f5c03f-d215-4609-a5cc-b1145f133bea-tls-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:54:11.440182 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.440204 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/36f5c03f-d215-4609-a5cc-b1145f133bea-dshm\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:54:11.844665 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.844628 2565 generic.go:356] "Generic (PLEG): container finished" podID="36f5c03f-d215-4609-a5cc-b1145f133bea" containerID="47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068" exitCode=0 Jul 21 17:54:11.844846 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.844675 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" Jul 21 17:54:11.844846 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.844715 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" event={"ID":"36f5c03f-d215-4609-a5cc-b1145f133bea","Type":"ContainerDied","Data":"47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068"} Jul 21 17:54:11.844846 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.844765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2" event={"ID":"36f5c03f-d215-4609-a5cc-b1145f133bea","Type":"ContainerDied","Data":"ac1a63c6cf857acdfb43e5e63b1fbb4be6edaa086be36b0bafc11088052c8943"} Jul 21 17:54:11.844846 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.844782 2565 scope.go:117] "RemoveContainer" containerID="47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068" Jul 21 17:54:11.853696 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.853679 2565 scope.go:117] "RemoveContainer" containerID="47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068" Jul 21 17:54:11.853974 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:54:11.853953 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068\": container with ID starting with 47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068 not found: ID does not exist" containerID="47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068" Jul 21 17:54:11.854031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.854012 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068"} err="failed to get container status \"47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068\": rpc error: code = NotFound desc = could not find container \"47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068\": container with ID starting with 47b1c2acbd197fdc464edb0be11dd6dbdfa2e68342900ea7cb08f5ab507e8068 not found: ID does not exist" Jul 21 17:54:11.864358 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.864336 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2"] Jul 21 17:54:11.868662 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:11.868641 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/auth-test-model-046b27-kserve-8fff85f5-rwdc2"] Jul 21 17:54:13.032183 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:13.032142 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36f5c03f-d215-4609-a5cc-b1145f133bea" path="/var/lib/kubelet/pods/36f5c03f-d215-4609-a5cc-b1145f133bea/volumes" Jul 21 17:54:27.929295 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.929266 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7"] Jul 21 17:54:27.929781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.929556 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36f5c03f-d215-4609-a5cc-b1145f133bea" containerName="main" Jul 21 17:54:27.929781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.929566 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="36f5c03f-d215-4609-a5cc-b1145f133bea" containerName="main" Jul 21 17:54:27.929781 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.929611 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="36f5c03f-d215-4609-a5cc-b1145f133bea" containerName="main" Jul 21 17:54:27.933564 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.933547 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:27.936798 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.936771 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"sub-test-model-046b27-kserve-self-signed-certs\"" Jul 21 17:54:27.936798 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.936792 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"default-dockercfg-89sxf\"" Jul 21 17:54:27.936995 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.936814 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"kube-root-ca.crt\"" Jul 21 17:54:27.936995 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.936872 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"openshift-service-ca.crt\"" Jul 21 17:54:27.941976 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.941950 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7"] Jul 21 17:54:27.981068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.981038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-home\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:27.981202 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.981079 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nmg2f\" (UniqueName: \"kubernetes.io/projected/87f4edee-37ee-4449-8377-3f2d0a60cdba-kube-api-access-nmg2f\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:27.981202 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.981138 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-dshm\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:27.981202 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.981176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87f4edee-37ee-4449-8377-3f2d0a60cdba-tls-certs\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:27.981202 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.981200 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-model-cache\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:27.981361 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:27.981216 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-tmp-dir\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.081896 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.081862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87f4edee-37ee-4449-8377-3f2d0a60cdba-tls-certs\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.081896 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.081895 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-model-cache\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.082098 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.082010 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-tmp-dir\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.082098 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.082064 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-home\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.082166 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.082105 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nmg2f\" (UniqueName: \"kubernetes.io/projected/87f4edee-37ee-4449-8377-3f2d0a60cdba-kube-api-access-nmg2f\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.082166 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.082132 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-dshm\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.082309 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.082290 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-model-cache\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.082388 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.082366 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-tmp-dir\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.082443 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.082412 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-home\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.084264 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.084224 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-dshm\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.084557 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.084536 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87f4edee-37ee-4449-8377-3f2d0a60cdba-tls-certs\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.090201 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.090178 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nmg2f\" (UniqueName: \"kubernetes.io/projected/87f4edee-37ee-4449-8377-3f2d0a60cdba-kube-api-access-nmg2f\") pod \"sub-test-model-046b27-kserve-658488f565-92db7\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.243591 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.243558 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:28.373773 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.373709 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7"] Jul 21 17:54:28.376593 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:54:28.376567 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87f4edee_37ee_4449_8377_3f2d0a60cdba.slice/crio-81aa576788f24972f92f383e54bd510b49b94a8102c2d73664e81ec420b0eb4a WatchSource:0}: Error finding container 81aa576788f24972f92f383e54bd510b49b94a8102c2d73664e81ec420b0eb4a: Status 404 returned error can't find the container with id 81aa576788f24972f92f383e54bd510b49b94a8102c2d73664e81ec420b0eb4a Jul 21 17:54:28.916478 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.916435 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" event={"ID":"87f4edee-37ee-4449-8377-3f2d0a60cdba","Type":"ContainerStarted","Data":"23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c"} Jul 21 17:54:28.916659 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.916487 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" event={"ID":"87f4edee-37ee-4449-8377-3f2d0a60cdba","Type":"ContainerStarted","Data":"81aa576788f24972f92f383e54bd510b49b94a8102c2d73664e81ec420b0eb4a"} Jul 21 17:54:28.934384 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:28.934331 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" podStartSLOduration=1.9343110540000001 podStartE2EDuration="1.934311054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:54:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:54:28.93357722 +0000 UTC m=+2768.547121522" watchObservedRunningTime="2026-07-21 17:54:28.934311054 +0000 UTC m=+2768.547855357" Jul 21 17:54:38.243920 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:38.243881 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:38.243920 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:38.243932 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:38.256457 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:38.256429 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:54:38.966078 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:54:38.966045 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:55:13.729175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:13.729143 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7"] Jul 21 17:55:28.749085 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:28.749035 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" podUID="87f4edee-37ee-4449-8377-3f2d0a60cdba" containerName="main" containerID="cri-o://23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c" gracePeriod=45 Jul 21 17:55:28.991191 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:28.991164 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:55:29.091919 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.091829 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-dshm\") pod \"87f4edee-37ee-4449-8377-3f2d0a60cdba\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " Jul 21 17:55:29.091919 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.091865 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-model-cache\") pod \"87f4edee-37ee-4449-8377-3f2d0a60cdba\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " Jul 21 17:55:29.091919 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.091919 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-tmp-dir\") pod \"87f4edee-37ee-4449-8377-3f2d0a60cdba\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " Jul 21 17:55:29.092223 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.091970 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nmg2f\" (UniqueName: \"kubernetes.io/projected/87f4edee-37ee-4449-8377-3f2d0a60cdba-kube-api-access-nmg2f\") pod \"87f4edee-37ee-4449-8377-3f2d0a60cdba\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " Jul 21 17:55:29.092223 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.092043 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-home\") pod \"87f4edee-37ee-4449-8377-3f2d0a60cdba\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " Jul 21 17:55:29.092223 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.092100 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87f4edee-37ee-4449-8377-3f2d0a60cdba-tls-certs\") pod \"87f4edee-37ee-4449-8377-3f2d0a60cdba\" (UID: \"87f4edee-37ee-4449-8377-3f2d0a60cdba\") " Jul 21 17:55:29.092411 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.092220 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-model-cache" (OuterVolumeSpecName: "model-cache") pod "87f4edee-37ee-4449-8377-3f2d0a60cdba" (UID: "87f4edee-37ee-4449-8377-3f2d0a60cdba"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:55:29.092411 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.092261 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "87f4edee-37ee-4449-8377-3f2d0a60cdba" (UID: "87f4edee-37ee-4449-8377-3f2d0a60cdba"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:55:29.092411 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.092386 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-home" (OuterVolumeSpecName: "home") pod "87f4edee-37ee-4449-8377-3f2d0a60cdba" (UID: "87f4edee-37ee-4449-8377-3f2d0a60cdba"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:55:29.092411 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.092412 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-model-cache\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:55:29.092669 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.092427 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-tmp-dir\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:55:29.094160 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.094133 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/87f4edee-37ee-4449-8377-3f2d0a60cdba-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "87f4edee-37ee-4449-8377-3f2d0a60cdba" (UID: "87f4edee-37ee-4449-8377-3f2d0a60cdba"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:55:29.094295 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.094186 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-dshm" (OuterVolumeSpecName: "dshm") pod "87f4edee-37ee-4449-8377-3f2d0a60cdba" (UID: "87f4edee-37ee-4449-8377-3f2d0a60cdba"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:55:29.094295 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.094189 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87f4edee-37ee-4449-8377-3f2d0a60cdba-kube-api-access-nmg2f" (OuterVolumeSpecName: "kube-api-access-nmg2f") pod "87f4edee-37ee-4449-8377-3f2d0a60cdba" (UID: "87f4edee-37ee-4449-8377-3f2d0a60cdba"). InnerVolumeSpecName "kube-api-access-nmg2f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:55:29.167206 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.167168 2565 generic.go:356] "Generic (PLEG): container finished" podID="87f4edee-37ee-4449-8377-3f2d0a60cdba" containerID="23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c" exitCode=0 Jul 21 17:55:29.167206 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.167209 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" Jul 21 17:55:29.167467 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.167273 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" event={"ID":"87f4edee-37ee-4449-8377-3f2d0a60cdba","Type":"ContainerDied","Data":"23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c"} Jul 21 17:55:29.167467 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.167316 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" event={"ID":"87f4edee-37ee-4449-8377-3f2d0a60cdba","Type":"ContainerDied","Data":"81aa576788f24972f92f383e54bd510b49b94a8102c2d73664e81ec420b0eb4a"} Jul 21 17:55:29.167467 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.167334 2565 scope.go:117] "RemoveContainer" containerID="23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c" Jul 21 17:55:29.175650 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.175632 2565 scope.go:117] "RemoveContainer" containerID="23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c" Jul 21 17:55:29.175883 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:55:29.175863 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c\": container with ID starting with 23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c not found: ID does not exist" containerID="23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c" Jul 21 17:55:29.175949 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.175895 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c"} err="failed to get container status \"23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c\": rpc error: code = NotFound desc = could not find container \"23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c\": container with ID starting with 23b8856295c58d57909d54202655d7d8e54bb7ebb387af879890de75ece3093c not found: ID does not exist" Jul 21 17:55:29.188015 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.187990 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7"] Jul 21 17:55:29.192659 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.192631 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7"] Jul 21 17:55:29.192962 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.192945 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-dshm\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:55:29.193007 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.192965 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nmg2f\" (UniqueName: \"kubernetes.io/projected/87f4edee-37ee-4449-8377-3f2d0a60cdba-kube-api-access-nmg2f\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:55:29.193007 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.192975 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/87f4edee-37ee-4449-8377-3f2d0a60cdba-home\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:55:29.193007 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:29.192984 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/87f4edee-37ee-4449-8377-3f2d0a60cdba-tls-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:55:31.032075 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:31.032035 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87f4edee-37ee-4449-8377-3f2d0a60cdba" path="/var/lib/kubelet/pods/87f4edee-37ee-4449-8377-3f2d0a60cdba/volumes" Jul 21 17:55:33.956491 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:33.956447 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-shared-b-046b27/sub-test-model-046b27-kserve-658488f565-92db7" podUID="87f4edee-37ee-4449-8377-3f2d0a60cdba" containerName="main" probeResult="failure" output="Get \"https://10.133.0.38:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Jul 21 17:55:39.721028 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.720994 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn"] Jul 21 17:55:39.721784 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.721735 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87f4edee-37ee-4449-8377-3f2d0a60cdba" containerName="main" Jul 21 17:55:39.721784 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.721774 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="87f4edee-37ee-4449-8377-3f2d0a60cdba" containerName="main" Jul 21 17:55:39.723689 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.722071 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="87f4edee-37ee-4449-8377-3f2d0a60cdba" containerName="main" Jul 21 17:55:39.729906 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.729885 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.732272 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.732251 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"openshift-service-ca.crt\"" Jul 21 17:55:39.732397 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.732254 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"kube-root-ca.crt\"" Jul 21 17:55:39.733260 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.733215 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"rate-test-model-046b27-kserve-self-signed-certs\"" Jul 21 17:55:39.733677 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.733216 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-046b27\"/\"default-dockercfg-89sxf\"" Jul 21 17:55:39.735841 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.735821 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn"] Jul 21 17:55:39.780531 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.780495 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-dshm\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.780682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.780542 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tmp-dir\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.780682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.780568 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-home\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.780682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.780598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-model-cache\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.780682 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.780621 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tls-certs\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.780824 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.780741 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc9gn\" (UniqueName: \"kubernetes.io/projected/fadc0b65-deab-4f1a-bad6-d5d79acd3619-kube-api-access-vc9gn\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882035 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882003 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tmp-dir\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882035 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882036 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-home\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882267 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882056 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-model-cache\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882267 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882110 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tls-certs\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882267 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882213 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vc9gn\" (UniqueName: \"kubernetes.io/projected/fadc0b65-deab-4f1a-bad6-d5d79acd3619-kube-api-access-vc9gn\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882420 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882282 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-dshm\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882479 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882449 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-home\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882479 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882464 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-model-cache\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.882583 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.882540 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tmp-dir\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.884519 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.884499 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-dshm\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.884800 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.884781 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tls-certs\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:39.891439 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:39.891414 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc9gn\" (UniqueName: \"kubernetes.io/projected/fadc0b65-deab-4f1a-bad6-d5d79acd3619-kube-api-access-vc9gn\") pod \"rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:40.042208 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:40.042166 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:40.168852 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:40.168825 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn"] Jul 21 17:55:40.171056 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:55:40.171014 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfadc0b65_deab_4f1a_bad6_d5d79acd3619.slice/crio-94e6911b531ab1ca9bf658d38192a99843e7b47e29a88360e46309bdbaef70eb WatchSource:0}: Error finding container 94e6911b531ab1ca9bf658d38192a99843e7b47e29a88360e46309bdbaef70eb: Status 404 returned error can't find the container with id 94e6911b531ab1ca9bf658d38192a99843e7b47e29a88360e46309bdbaef70eb Jul 21 17:55:40.211864 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:40.211834 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" event={"ID":"fadc0b65-deab-4f1a-bad6-d5d79acd3619","Type":"ContainerStarted","Data":"94e6911b531ab1ca9bf658d38192a99843e7b47e29a88360e46309bdbaef70eb"} Jul 21 17:55:41.218542 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:41.218507 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" event={"ID":"fadc0b65-deab-4f1a-bad6-d5d79acd3619","Type":"ContainerStarted","Data":"258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2"} Jul 21 17:55:41.236573 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:41.236525 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" podStartSLOduration=2.236510753 podStartE2EDuration="2.236510753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-21 17:55:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:55:41.235407206 +0000 UTC m=+2840.848951505" watchObservedRunningTime="2026-07-21 17:55:41.236510753 +0000 UTC m=+2840.850055064" Jul 21 17:55:50.042612 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:50.042559 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:50.042612 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:50.042618 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:50.055093 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:50.055058 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:55:50.264427 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:55:50.264400 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:56:18.799680 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:18.799590 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn"] Jul 21 17:56:33.820355 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:33.820294 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" podUID="fadc0b65-deab-4f1a-bad6-d5d79acd3619" containerName="main" containerID="cri-o://258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2" gracePeriod=45 Jul 21 17:56:34.059145 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.059115 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:56:34.129894 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.129811 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vc9gn\" (UniqueName: \"kubernetes.io/projected/fadc0b65-deab-4f1a-bad6-d5d79acd3619-kube-api-access-vc9gn\") pod \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " Jul 21 17:56:34.129894 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.129863 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tmp-dir\") pod \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " Jul 21 17:56:34.129894 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.129892 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tls-certs\") pod \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " Jul 21 17:56:34.130180 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.129911 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-dshm\") pod \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " Jul 21 17:56:34.130180 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.129955 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-home\") pod \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " Jul 21 17:56:34.130180 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.129982 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-model-cache\") pod \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\" (UID: \"fadc0b65-deab-4f1a-bad6-d5d79acd3619\") " Jul 21 17:56:34.130381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.130222 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fadc0b65-deab-4f1a-bad6-d5d79acd3619" (UID: "fadc0b65-deab-4f1a-bad6-d5d79acd3619"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:56:34.130381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.130271 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-model-cache" (OuterVolumeSpecName: "model-cache") pod "fadc0b65-deab-4f1a-bad6-d5d79acd3619" (UID: "fadc0b65-deab-4f1a-bad6-d5d79acd3619"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:56:34.130381 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.130328 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-home" (OuterVolumeSpecName: "home") pod "fadc0b65-deab-4f1a-bad6-d5d79acd3619" (UID: "fadc0b65-deab-4f1a-bad6-d5d79acd3619"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:56:34.130522 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.130383 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-model-cache\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:56:34.130522 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.130427 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tmp-dir\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:56:34.132097 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.132074 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fadc0b65-deab-4f1a-bad6-d5d79acd3619" (UID: "fadc0b65-deab-4f1a-bad6-d5d79acd3619"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 21 17:56:34.132097 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.132086 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-dshm" (OuterVolumeSpecName: "dshm") pod "fadc0b65-deab-4f1a-bad6-d5d79acd3619" (UID: "fadc0b65-deab-4f1a-bad6-d5d79acd3619"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:56:34.132193 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.132090 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fadc0b65-deab-4f1a-bad6-d5d79acd3619-kube-api-access-vc9gn" (OuterVolumeSpecName: "kube-api-access-vc9gn") pod "fadc0b65-deab-4f1a-bad6-d5d79acd3619" (UID: "fadc0b65-deab-4f1a-bad6-d5d79acd3619"). InnerVolumeSpecName "kube-api-access-vc9gn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:56:34.231344 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.231314 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vc9gn\" (UniqueName: \"kubernetes.io/projected/fadc0b65-deab-4f1a-bad6-d5d79acd3619-kube-api-access-vc9gn\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:56:34.231344 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.231338 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadc0b65-deab-4f1a-bad6-d5d79acd3619-tls-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:56:34.231344 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.231347 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-dshm\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:56:34.231556 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.231356 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadc0b65-deab-4f1a-bad6-d5d79acd3619-home\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:56:34.436218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.436129 2565 generic.go:356] "Generic (PLEG): container finished" podID="fadc0b65-deab-4f1a-bad6-d5d79acd3619" containerID="258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2" exitCode=0 Jul 21 17:56:34.436218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.436172 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" Jul 21 17:56:34.436218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.436200 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" event={"ID":"fadc0b65-deab-4f1a-bad6-d5d79acd3619","Type":"ContainerDied","Data":"258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2"} Jul 21 17:56:34.436475 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.436255 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn" event={"ID":"fadc0b65-deab-4f1a-bad6-d5d79acd3619","Type":"ContainerDied","Data":"94e6911b531ab1ca9bf658d38192a99843e7b47e29a88360e46309bdbaef70eb"} Jul 21 17:56:34.436475 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.436274 2565 scope.go:117] "RemoveContainer" containerID="258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2" Jul 21 17:56:34.445223 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.445203 2565 scope.go:117] "RemoveContainer" containerID="258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2" Jul 21 17:56:34.445506 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:56:34.445485 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2\": container with ID starting with 258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2 not found: ID does not exist" containerID="258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2" Jul 21 17:56:34.445577 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.445513 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2"} err="failed to get container status \"258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2\": rpc error: code = NotFound desc = could not find container \"258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2\": container with ID starting with 258a4b2288a6c129bc4007df3f60b18ab4ee2399311d826505e45e66055230b2 not found: ID does not exist" Jul 21 17:56:34.457434 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.457389 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn"] Jul 21 17:56:34.459422 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:34.459401 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-046b27/rate-test-model-046b27-kserve-5cd99cfcf9-h6zxn"] Jul 21 17:56:35.031500 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:35.031470 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fadc0b65-deab-4f1a-bad6-d5d79acd3619" path="/var/lib/kubelet/pods/fadc0b65-deab-4f1a-bad6-d5d79acd3619/volumes" Jul 21 17:56:59.656394 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:59.656361 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm"] Jul 21 17:56:59.656887 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:56:59.656595 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" containerID="cri-o://a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863" gracePeriod=30 Jul 21 17:57:04.903139 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.903115 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:57:04.979194 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979129 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-socket\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979194 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979160 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-envoy\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979194 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979179 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-certs\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979197 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-data\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979219 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-podinfo\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979273 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqcgf\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-kube-api-access-fqcgf\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979303 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-credential-socket\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979374 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-token\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979399 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istiod-ca-cert\") pod \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\" (UID: \"af0f3057-ebeb-4a36-9ce9-2617b08b180c\") " Jul 21 17:57:04.979862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979463 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:04.979862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979497 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:04.979862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979630 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:04.979862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979678 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:04.979862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979697 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:04.979862 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.979792 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:57:04.980171 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.980037 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-data" (OuterVolumeSpecName: "istio-data") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:04.981545 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.981510 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-token" (OuterVolumeSpecName: "istio-token") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:57:04.981642 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.981536 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-kube-api-access-fqcgf" (OuterVolumeSpecName: "kube-api-access-fqcgf") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "kube-api-access-fqcgf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:57:04.981642 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.981556 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:57:04.981642 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:04.981593 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "af0f3057-ebeb-4a36-9ce9-2617b08b180c" (UID: "af0f3057-ebeb-4a36-9ce9-2617b08b180c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:05.080889 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.080861 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:05.080889 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.080887 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:05.081022 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.080896 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:05.081022 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.080905 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqcgf\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-kube-api-access-fqcgf\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:05.081022 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.080916 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/af0f3057-ebeb-4a36-9ce9-2617b08b180c-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:05.081022 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.080924 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:05.081022 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.080932 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/af0f3057-ebeb-4a36-9ce9-2617b08b180c-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:05.562031 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.561999 2565 generic.go:356] "Generic (PLEG): container finished" podID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerID="a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863" exitCode=0 Jul 21 17:57:05.562257 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.562052 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" Jul 21 17:57:05.562257 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.562052 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" event={"ID":"af0f3057-ebeb-4a36-9ce9-2617b08b180c","Type":"ContainerDied","Data":"a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863"} Jul 21 17:57:05.562257 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.562183 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm" event={"ID":"af0f3057-ebeb-4a36-9ce9-2617b08b180c","Type":"ContainerDied","Data":"be1d0e900a4052e9a76a8eda7a631383900f1aa7bab6234c6e24e36193abbf88"} Jul 21 17:57:05.562257 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.562213 2565 scope.go:117] "RemoveContainer" containerID="a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863" Jul 21 17:57:05.573128 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.573104 2565 scope.go:117] "RemoveContainer" containerID="10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0" Jul 21 17:57:05.580265 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.580229 2565 scope.go:117] "RemoveContainer" containerID="a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863" Jul 21 17:57:05.580505 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:57:05.580481 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863\": container with ID starting with a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863 not found: ID does not exist" containerID="a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863" Jul 21 17:57:05.580580 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.580515 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863"} err="failed to get container status \"a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863\": rpc error: code = NotFound desc = could not find container \"a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863\": container with ID starting with a093978ee1728248e7f712f3c051224e5f4bf9faf40649512184628bf566d863 not found: ID does not exist" Jul 21 17:57:05.580580 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.580539 2565 scope.go:117] "RemoveContainer" containerID="10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0" Jul 21 17:57:05.580813 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:57:05.580788 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0\": container with ID starting with 10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0 not found: ID does not exist" containerID="10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0" Jul 21 17:57:05.580892 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.580817 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0"} err="failed to get container status \"10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0\": rpc error: code = NotFound desc = could not find container \"10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0\": container with ID starting with 10082adb903027c8d1ff388d3c2e1c86f69a91d68d18ffe4c56cca9a4f2a4bc0 not found: ID does not exist" Jul 21 17:57:05.583002 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.582979 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm"] Jul 21 17:57:05.586857 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:05.586838 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-43413b-openshift-default-574bdbdcf9-2twxm"] Jul 21 17:57:07.031984 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:07.031944 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" path="/var/lib/kubelet/pods/af0f3057-ebeb-4a36-9ce9-2617b08b180c/volumes" Jul 21 17:57:21.753073 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:21.753039 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm"] Jul 21 17:57:21.753507 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:21.753275 2565 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" containerID="cri-o://7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903" gracePeriod=30 Jul 21 17:57:27.000831 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.000805 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:57:27.066567 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066480 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istiod-ca-cert\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066567 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066524 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-socket\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066787 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-envoy\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066787 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066587 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-data\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066787 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066605 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-podinfo\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066787 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066646 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-certs\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066787 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066674 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-credential-socket\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066787 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066703 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-token\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.066787 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066728 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-clbgz\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-kube-api-access-clbgz\") pod \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\" (UID: \"b950a3b0-9b89-4d66-8d08-ab78a4477efe\") " Jul 21 17:57:27.067154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066926 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 21 17:57:27.067154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.066951 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:27.067154 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.067140 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:27.067350 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.067211 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:27.067864 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.067827 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-data" (OuterVolumeSpecName: "istio-data") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:27.068937 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.068906 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 21 17:57:27.069062 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.069039 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-kube-api-access-clbgz" (OuterVolumeSpecName: "kube-api-access-clbgz") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "kube-api-access-clbgz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:57:27.069106 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.069065 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 21 17:57:27.069106 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.069074 2565 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-token" (OuterVolumeSpecName: "istio-token") pod "b950a3b0-9b89-4d66-8d08-ab78a4477efe" (UID: "b950a3b0-9b89-4d66-8d08-ab78a4477efe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 21 17:57:27.167540 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167501 2565 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istiod-ca-cert\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167540 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167537 2565 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167540 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167547 2565 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-envoy\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167540 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167555 2565 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-data\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167563 2565 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-podinfo\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167571 2565 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-workload-certs\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167579 2565 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b950a3b0-9b89-4d66-8d08-ab78a4477efe-credential-socket\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167588 2565 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-istio-token\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.167775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.167596 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-clbgz\" (UniqueName: \"kubernetes.io/projected/b950a3b0-9b89-4d66-8d08-ab78a4477efe-kube-api-access-clbgz\") on node \"ip-10-0-142-109.ec2.internal\" DevicePath \"\"" Jul 21 17:57:27.650775 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.650736 2565 generic.go:356] "Generic (PLEG): container finished" podID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerID="7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903" exitCode=0 Jul 21 17:57:27.650948 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.650788 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" Jul 21 17:57:27.650948 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.650802 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" event={"ID":"b950a3b0-9b89-4d66-8d08-ab78a4477efe","Type":"ContainerDied","Data":"7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903"} Jul 21 17:57:27.650948 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.650848 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm" event={"ID":"b950a3b0-9b89-4d66-8d08-ab78a4477efe","Type":"ContainerDied","Data":"97622ec1ed802f9ad695953fd1f5f46b07cae228d7965d1df8b9a7876da348bd"} Jul 21 17:57:27.650948 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.650870 2565 scope.go:117] "RemoveContainer" containerID="7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903" Jul 21 17:57:27.661722 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.661705 2565 scope.go:117] "RemoveContainer" containerID="650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487" Jul 21 17:57:27.669524 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.669507 2565 scope.go:117] "RemoveContainer" containerID="7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903" Jul 21 17:57:27.669777 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:57:27.669745 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903\": container with ID starting with 7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903 not found: ID does not exist" containerID="7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903" Jul 21 17:57:27.669827 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.669783 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903"} err="failed to get container status \"7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903\": rpc error: code = NotFound desc = could not find container \"7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903\": container with ID starting with 7a1c495c68201b107babd28c58795f07efe85dc43d929c459bb954d4a6cef903 not found: ID does not exist" Jul 21 17:57:27.669827 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.669799 2565 scope.go:117] "RemoveContainer" containerID="650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487" Jul 21 17:57:27.670016 ip-10-0-142-109 kubenswrapper[2565]: E0721 17:57:27.669998 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487\": container with ID starting with 650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487 not found: ID does not exist" containerID="650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487" Jul 21 17:57:27.670068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.670025 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487"} err="failed to get container status \"650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487\": rpc error: code = NotFound desc = could not find container \"650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487\": container with ID starting with 650d676403a7986c181d04def228053263c710f86957668f6c7d53cfe765c487 not found: ID does not exist" Jul 21 17:57:27.675989 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.675961 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm"] Jul 21 17:57:27.679068 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:27.679050 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-046b27-openshift-default-5c6bd8b779-kc2zm"] Jul 21 17:57:28.371851 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:28.371823 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fpb9n_60057d50-b81d-40c7-b5e3-9e84050370f9/server/0.log" Jul 21 17:57:28.854488 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:28.854463 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b498688-wgbwp_010c7fb0-4f11-4158-a9df-70ccf23fc509/copy-manifests/0.log" Jul 21 17:57:28.872750 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:28.872724 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b498688-wgbwp_010c7fb0-4f11-4158-a9df-70ccf23fc509/manager/0.log" Jul 21 17:57:29.031660 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:29.031624 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" path="/var/lib/kubelet/pods/b950a3b0-9b89-4d66-8d08-ab78a4477efe/volumes" Jul 21 17:57:29.591584 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:29.591546 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-kzh2v_4075c18e-1137-4749-859f-2b30253d2fa0/postgres/0.log" Jul 21 17:57:32.177378 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:32.177345 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgh9s_fde8b9dd-5666-4d6e-9d9a-f9791a014f1f/istio-proxy/0.log" Jul 21 17:57:32.301045 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:32.301014 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2v9bs_8a21adf3-2640-4112-857b-740b6f155b60/discovery/0.log" Jul 21 17:57:33.856404 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:33.856376 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rnkwn_d6fed8dd-4022-47d3-aeeb-0811dab4b48a/main/0.log" Jul 21 17:57:42.833979 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:42.833898 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-slspc_ab2c4f15-76d5-4e23-81e8-5bcce1bfe0f4/global-pull-secret-syncer/0.log" Jul 21 17:57:42.995772 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:42.995735 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7f57l_8b818316-6b2d-4e37-b96e-ff47692a0fd9/konnectivity-agent/0.log" Jul 21 17:57:43.186804 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:43.186726 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-109.ec2.internal_70278ed5d98e32abf177b8a325b5bf17/haproxy/0.log" Jul 21 17:57:50.964612 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:50.964580 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-k9qgp_05af373c-9955-4714-9604-4e7903dd202f/monitoring-plugin/0.log" Jul 21 17:57:51.153911 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:51.153880 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzzj8_743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1/node-exporter/0.log" Jul 21 17:57:51.173914 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:51.173889 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzzj8_743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1/kube-rbac-proxy/0.log" Jul 21 17:57:51.195273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:51.195253 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fzzj8_743c28b5-25dc-4a56-bc2a-9cd7f5c57cb1/init-textfile/0.log" Jul 21 17:57:51.468991 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:51.468956 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-rz5z2_177c485f-7bfa-464f-b941-254d609f4121/prometheus-operator/0.log" Jul 21 17:57:51.485366 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:51.485341 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-rz5z2_177c485f-7bfa-464f-b941-254d609f4121/kube-rbac-proxy/0.log" Jul 21 17:57:52.661465 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661430 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7"] Jul 21 17:57:52.661908 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661833 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" Jul 21 17:57:52.661908 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661851 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" Jul 21 17:57:52.661908 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661864 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" Jul 21 17:57:52.661908 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661873 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" Jul 21 17:57:52.661908 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661890 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" Jul 21 17:57:52.661908 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661900 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" Jul 21 17:57:52.662218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661917 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fadc0b65-deab-4f1a-bad6-d5d79acd3619" containerName="main" Jul 21 17:57:52.662218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.661926 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="fadc0b65-deab-4f1a-bad6-d5d79acd3619" containerName="main" Jul 21 17:57:52.662218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.662008 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="af0f3057-ebeb-4a36-9ce9-2617b08b180c" containerName="istio-proxy" Jul 21 17:57:52.662218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.662019 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" Jul 21 17:57:52.662218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.662028 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b950a3b0-9b89-4d66-8d08-ab78a4477efe" containerName="istio-proxy" Jul 21 17:57:52.662218 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.662037 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="fadc0b65-deab-4f1a-bad6-d5d79acd3619" containerName="main" Jul 21 17:57:52.664996 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.664981 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.667463 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.667441 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rjthx\"/\"kube-root-ca.crt\"" Jul 21 17:57:52.667587 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.667442 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rjthx\"/\"openshift-service-ca.crt\"" Jul 21 17:57:52.668098 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.668055 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rjthx\"/\"default-dockercfg-d9sl7\"" Jul 21 17:57:52.673472 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.673451 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7"] Jul 21 17:57:52.774522 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.774487 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c76jc\" (UniqueName: \"kubernetes.io/projected/46138857-8ed2-4700-ad6b-15f240508274-kube-api-access-c76jc\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.774696 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.774566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-podres\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.774696 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.774587 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-sys\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.774696 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.774605 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-proc\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.774696 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.774623 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-lib-modules\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.874970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.874940 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-podres\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.874970 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.874973 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-sys\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.875175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.874993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-proc\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.875175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.875011 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-lib-modules\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.875175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.875044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c76jc\" (UniqueName: \"kubernetes.io/projected/46138857-8ed2-4700-ad6b-15f240508274-kube-api-access-c76jc\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.875175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.875082 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-sys\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.875175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.875083 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-proc\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.875175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.875112 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-podres\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.875175 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.875149 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/46138857-8ed2-4700-ad6b-15f240508274-lib-modules\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.883822 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.883804 2565 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c76jc\" (UniqueName: \"kubernetes.io/projected/46138857-8ed2-4700-ad6b-15f240508274-kube-api-access-c76jc\") pod \"perf-node-gather-daemonset-flsj7\" (UID: \"46138857-8ed2-4700-ad6b-15f240508274\") " pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:52.974845 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:52.974777 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:53.100645 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:53.100616 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7"] Jul 21 17:57:53.103559 ip-10-0-142-109 kubenswrapper[2565]: W0721 17:57:53.103532 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod46138857_8ed2_4700_ad6b_15f240508274.slice/crio-8041069f6f5613d422ac6ef02dde82c46a17fe6f8b4738a8d55b7ee80c5fb970 WatchSource:0}: Error finding container 8041069f6f5613d422ac6ef02dde82c46a17fe6f8b4738a8d55b7ee80c5fb970: Status 404 returned error can't find the container with id 8041069f6f5613d422ac6ef02dde82c46a17fe6f8b4738a8d55b7ee80c5fb970 Jul 21 17:57:53.755273 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:53.755226 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" event={"ID":"46138857-8ed2-4700-ad6b-15f240508274","Type":"ContainerStarted","Data":"a85ac02c18992b78c5f22c22ad1eca6c7dc7d02c81e7c5b1b0f7bb7772879911"} Jul 21 17:57:53.755754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:53.755288 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" event={"ID":"46138857-8ed2-4700-ad6b-15f240508274","Type":"ContainerStarted","Data":"8041069f6f5613d422ac6ef02dde82c46a17fe6f8b4738a8d55b7ee80c5fb970"} Jul 21 17:57:53.755754 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:53.755354 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:57:53.772656 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:53.772613 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" podStartSLOduration=1.7725977880000001 podStartE2EDuration="1.772597788s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-21 17:57:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-21 17:57:53.771381573 +0000 UTC m=+2973.384925875" watchObservedRunningTime="2026-07-21 17:57:53.772597788 +0000 UTC m=+2973.386142088" Jul 21 17:57:55.748029 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:55.747996 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2pfj_01e78e36-df87-42ea-91d9-d661ade09e86/dns/0.log" Jul 21 17:57:55.768525 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:55.768501 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b2pfj_01e78e36-df87-42ea-91d9-d661ade09e86/kube-rbac-proxy/0.log" Jul 21 17:57:55.836265 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:55.836228 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5vmr7_013607f3-00b2-48cb-9e89-d8a3cddafd22/dns-node-resolver/0.log" Jul 21 17:57:56.495476 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:56.495441 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-wqfb5_d49b95fd-0f47-423f-a36a-536f9ef79bf9/node-ca/0.log" Jul 21 17:57:57.400452 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:57.400420 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgh9s_fde8b9dd-5666-4d6e-9d9a-f9791a014f1f/istio-proxy/0.log" Jul 21 17:57:57.623959 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:57.623924 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2v9bs_8a21adf3-2640-4112-857b-740b6f155b60/discovery/0.log" Jul 21 17:57:58.575883 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:58.575856 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fsxck_c21775ac-3111-443d-a5cb-b9d7641e6e56/serve-healthcheck-canary/0.log" Jul 21 17:57:59.147770 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:59.147739 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m4qjh_b3e8a413-7456-422e-a38b-f29321c499a0/kube-rbac-proxy/0.log" Jul 21 17:57:59.171691 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:59.171668 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m4qjh_b3e8a413-7456-422e-a38b-f29321c499a0/exporter/0.log" Jul 21 17:57:59.195229 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:59.195204 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m4qjh_b3e8a413-7456-422e-a38b-f29321c499a0/extractor/0.log" Jul 21 17:57:59.768145 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:57:59.768117 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rjthx/perf-node-gather-daemonset-flsj7" Jul 21 17:58:01.779303 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:01.779271 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-fpb9n_60057d50-b81d-40c7-b5e3-9e84050370f9/server/0.log" Jul 21 17:58:02.088753 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:02.088659 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b498688-wgbwp_010c7fb0-4f11-4158-a9df-70ccf23fc509/manager/0.log" Jul 21 17:58:02.109212 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:02.109188 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-65b498688-wgbwp_010c7fb0-4f11-4158-a9df-70ccf23fc509/copy-manifests/0.log" Jul 21 17:58:03.119424 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:03.119376 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgh9s_fde8b9dd-5666-4d6e-9d9a-f9791a014f1f/istio-proxy/0.log" Jul 21 17:58:03.334636 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:03.334558 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2v9bs_8a21adf3-2640-4112-857b-740b6f155b60/discovery/0.log" Jul 21 17:58:03.444544 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:03.444516 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-646df98c5d-p9wtt_ccf27e6b-dc1e-4da1-a6bf-383d43b02328/manager/0.log" Jul 21 17:58:04.784612 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:04.784589 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4lgh9s_fde8b9dd-5666-4d6e-9d9a-f9791a014f1f/istio-proxy/0.log" Jul 21 17:58:05.007205 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:05.007168 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-2v9bs_8a21adf3-2640-4112-857b-740b6f155b60/discovery/0.log" Jul 21 17:58:10.694003 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.693972 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xhljt_d62be561-533b-4be0-8ab5-6ea408150db5/kube-multus-additional-cni-plugins/0.log" Jul 21 17:58:10.738884 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.738851 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xhljt_d62be561-533b-4be0-8ab5-6ea408150db5/egress-router-binary-copy/0.log" Jul 21 17:58:10.771543 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.771510 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xhljt_d62be561-533b-4be0-8ab5-6ea408150db5/cni-plugins/0.log" Jul 21 17:58:10.813653 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.813621 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xhljt_d62be561-533b-4be0-8ab5-6ea408150db5/bond-cni-plugin/0.log" Jul 21 17:58:10.850897 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.850870 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xhljt_d62be561-533b-4be0-8ab5-6ea408150db5/routeoverride-cni/0.log" Jul 21 17:58:10.881664 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.881639 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xhljt_d62be561-533b-4be0-8ab5-6ea408150db5/whereabouts-cni-bincopy/0.log" Jul 21 17:58:10.909226 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.909198 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-xhljt_d62be561-533b-4be0-8ab5-6ea408150db5/whereabouts-cni/0.log" Jul 21 17:58:10.950805 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:10.950730 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wjfzj_b4d06660-fbc8-4804-96be-69563b12276d/kube-multus/0.log" Jul 21 17:58:11.146280 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:11.146230 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xrl6x_17870260-a36e-451c-93ac-56c19cdb45c7/network-metrics-daemon/0.log" Jul 21 17:58:11.172771 ip-10-0-142-109 kubenswrapper[2565]: I0721 17:58:11.172746 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xrl6x_17870260-a36e-451c-93ac-56c19cdb45c7/kube-rbac-proxy/0.log"