Jul 27 23:34:42.684224 ip-10-0-142-203 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 27 23:34:42.684237 ip-10-0-142-203 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 27 23:34:42.684247 ip-10-0-142-203 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 27 23:34:42.684572 ip-10-0-142-203 systemd[1]: Failed to start Kubernetes Kubelet. Jul 27 23:34:51.903458 ip-10-0-142-203 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 27 23:34:51.903473 ip-10-0-142-203 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot e37c18768430428898e18c65fb2f0556 -- Jul 27 23:36:37.783933 ip-10-0-142-203 systemd[1]: Starting Kubernetes Kubelet... Jul 27 23:36:38.091755 ip-10-0-142-203 kubenswrapper[2561]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 23:36:38.091755 ip-10-0-142-203 kubenswrapper[2561]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 27 23:36:38.091755 ip-10-0-142-203 kubenswrapper[2561]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 23:36:38.091755 ip-10-0-142-203 kubenswrapper[2561]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 27 23:36:38.091755 ip-10-0-142-203 kubenswrapper[2561]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 27 23:36:38.093069 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.092984 2561 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 27 23:36:38.098272 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098256 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:36:38.098272 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098272 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098276 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098279 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098282 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098285 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098288 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098292 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098295 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098298 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098300 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098303 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098306 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098309 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098311 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098314 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098317 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098319 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098322 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098324 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098327 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:36:38.098335 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098330 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098332 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098335 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098338 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098340 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098343 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098346 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098349 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098352 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098354 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098357 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098359 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098362 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098364 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098367 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098369 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098372 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098375 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098378 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098380 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:36:38.098823 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098383 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098386 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098390 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098392 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098395 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098397 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098399 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098402 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098405 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098407 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098410 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098412 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098414 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098417 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098420 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098423 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098425 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098428 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098431 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098433 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:36:38.099307 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098436 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098439 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098441 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098444 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098447 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098449 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098452 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098454 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098457 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098460 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098462 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098465 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098469 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098473 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098476 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098479 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098482 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098485 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098487 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:36:38.099808 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098490 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098493 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098495 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098498 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098501 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098504 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098507 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098510 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098513 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098516 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098520 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098524 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098528 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098531 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098534 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098537 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098540 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098542 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.098545 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:36:38.100264 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099242 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099249 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099252 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099256 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099259 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099262 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099265 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099267 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099270 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099273 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099275 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099278 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099280 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099283 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099286 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099288 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099290 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099293 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099296 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099298 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:36:38.100733 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099301 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099304 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099307 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099310 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099313 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099315 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099318 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099321 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099324 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099327 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099329 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099332 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099334 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099337 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099339 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099342 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099344 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099346 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099349 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:36:38.101221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099352 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099355 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099358 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099360 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099365 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099368 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099371 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099374 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099377 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099379 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099382 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099385 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099388 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099392 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099394 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099398 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099400 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099403 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099406 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099410 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:36:38.101699 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099412 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099415 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099418 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099420 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099423 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099425 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099428 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099430 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099432 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099436 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099440 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099443 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099447 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099450 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099452 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099455 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099458 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099460 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099463 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:36:38.102200 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099466 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099468 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099471 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099476 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099479 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099481 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099485 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099488 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099490 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099493 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099496 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099499 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099501 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099504 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099506 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099509 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099511 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099514 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099516 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099519 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:36:38.102655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099522 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099910 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099916 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099920 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099923 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099931 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099934 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099937 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099939 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099942 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099945 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099947 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099950 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099952 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099954 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099957 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099960 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099963 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099966 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099969 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:36:38.103177 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099973 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099976 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099979 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099982 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099985 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099988 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099991 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099993 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099996 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.099998 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100001 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100003 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100006 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100009 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100011 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100014 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100016 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100020 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100024 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:36:38.103693 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100027 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100029 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100032 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100035 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100037 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100040 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100043 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100045 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100048 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100050 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100053 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100055 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100058 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100061 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100063 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100065 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100068 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100071 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100073 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100077 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:36:38.104144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100079 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100082 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100084 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100086 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100089 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100091 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100095 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100099 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100101 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100104 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100107 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100110 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100113 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100115 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100118 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100121 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100124 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100128 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100130 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:36:38.104665 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100133 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100135 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100138 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100140 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100143 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100146 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100148 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100150 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100153 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100155 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100158 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100160 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100163 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100165 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100168 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100170 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100172 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100175 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100177 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100180 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:36:38.105144 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100182 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100184 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100247 2561 flags.go:64] FLAG: --address="0.0.0.0" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100258 2561 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100264 2561 flags.go:64] FLAG: --anonymous-auth="true" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100268 2561 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100273 2561 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100276 2561 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100283 2561 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100291 2561 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100294 2561 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100297 2561 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100300 2561 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100303 2561 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100306 2561 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100309 2561 flags.go:64] FLAG: --cgroup-root="" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100312 2561 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100315 2561 flags.go:64] FLAG: --client-ca-file="" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100318 2561 flags.go:64] FLAG: --cloud-provider="external" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100321 2561 flags.go:64] FLAG: --cluster-dns="[]" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100325 2561 flags.go:64] FLAG: --cluster-domain="" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100327 2561 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100330 2561 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100333 2561 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 27 23:36:38.105655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100336 2561 flags.go:64] FLAG: --container-log-max-files="5" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100340 2561 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100343 2561 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100346 2561 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100349 2561 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100352 2561 flags.go:64] FLAG: --contention-profiling="false" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100356 2561 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100359 2561 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100362 2561 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100365 2561 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100369 2561 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100372 2561 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100375 2561 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100378 2561 flags.go:64] FLAG: --enable-load-reader="false" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100380 2561 flags.go:64] FLAG: --enable-server="true" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100384 2561 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100389 2561 flags.go:64] FLAG: --event-burst="100" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100392 2561 flags.go:64] FLAG: --event-qps="50" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100394 2561 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100397 2561 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100400 2561 flags.go:64] FLAG: --eviction-hard="" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100403 2561 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100407 2561 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100409 2561 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100412 2561 flags.go:64] FLAG: --eviction-soft="" Jul 27 23:36:38.106255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100415 2561 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100418 2561 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100421 2561 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100424 2561 flags.go:64] FLAG: --experimental-mounter-path="" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100426 2561 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100429 2561 flags.go:64] FLAG: --fail-swap-on="true" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100432 2561 flags.go:64] FLAG: --feature-gates="" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100436 2561 flags.go:64] FLAG: --file-check-frequency="20s" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100438 2561 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100442 2561 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100444 2561 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100447 2561 flags.go:64] FLAG: --healthz-port="10248" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100450 2561 flags.go:64] FLAG: --help="false" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100454 2561 flags.go:64] FLAG: --hostname-override="ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100457 2561 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100460 2561 flags.go:64] FLAG: --http-check-frequency="20s" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100463 2561 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100466 2561 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100469 2561 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100472 2561 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100475 2561 flags.go:64] FLAG: --image-service-endpoint="" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100477 2561 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100480 2561 flags.go:64] FLAG: --kube-api-burst="100" Jul 27 23:36:38.106925 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100485 2561 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100488 2561 flags.go:64] FLAG: --kube-api-qps="50" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100491 2561 flags.go:64] FLAG: --kube-reserved="" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100493 2561 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100496 2561 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100499 2561 flags.go:64] FLAG: --kubelet-cgroups="" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100502 2561 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100505 2561 flags.go:64] FLAG: --lock-file="" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100508 2561 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100511 2561 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100514 2561 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100519 2561 flags.go:64] FLAG: --log-json-split-stream="false" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100522 2561 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100525 2561 flags.go:64] FLAG: --log-text-split-stream="false" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100527 2561 flags.go:64] FLAG: --logging-format="text" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100530 2561 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100533 2561 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100536 2561 flags.go:64] FLAG: --manifest-url="" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100539 2561 flags.go:64] FLAG: --manifest-url-header="" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100543 2561 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100546 2561 flags.go:64] FLAG: --max-open-files="1000000" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100550 2561 flags.go:64] FLAG: --max-pods="110" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100554 2561 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100557 2561 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100560 2561 flags.go:64] FLAG: --memory-manager-policy="None" Jul 27 23:36:38.107521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100563 2561 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100566 2561 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100569 2561 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100571 2561 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100578 2561 flags.go:64] FLAG: --node-status-max-images="50" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100581 2561 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100584 2561 flags.go:64] FLAG: --oom-score-adj="-999" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100588 2561 flags.go:64] FLAG: --pod-cidr="" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100591 2561 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100597 2561 flags.go:64] FLAG: --pod-manifest-path="" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100600 2561 flags.go:64] FLAG: --pod-max-pids="-1" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100603 2561 flags.go:64] FLAG: --pods-per-core="0" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100606 2561 flags.go:64] FLAG: --port="10250" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100608 2561 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100611 2561 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00f66405ccf783550" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100614 2561 flags.go:64] FLAG: --qos-reserved="" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100617 2561 flags.go:64] FLAG: --read-only-port="10255" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100620 2561 flags.go:64] FLAG: --register-node="true" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100623 2561 flags.go:64] FLAG: --register-with-taints="" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100626 2561 flags.go:64] FLAG: --registry-burst="10" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100629 2561 flags.go:64] FLAG: --registry-qps="5" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100633 2561 flags.go:64] FLAG: --reserved-cpus="" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100635 2561 flags.go:64] FLAG: --reserved-memory="" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100639 2561 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100642 2561 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 27 23:36:38.108151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100645 2561 flags.go:64] FLAG: --rotate-certificates="false" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100647 2561 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100650 2561 flags.go:64] FLAG: --runonce="false" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100653 2561 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100657 2561 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100660 2561 flags.go:64] FLAG: --seccomp-default="false" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100663 2561 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100665 2561 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100682 2561 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100687 2561 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100692 2561 flags.go:64] FLAG: --storage-driver-password="root" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100696 2561 flags.go:64] FLAG: --storage-driver-secure="false" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100699 2561 flags.go:64] FLAG: --storage-driver-table="stats" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100702 2561 flags.go:64] FLAG: --storage-driver-user="root" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100704 2561 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100710 2561 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100713 2561 flags.go:64] FLAG: --system-cgroups="" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100716 2561 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100722 2561 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100724 2561 flags.go:64] FLAG: --tls-cert-file="" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100727 2561 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100732 2561 flags.go:64] FLAG: --tls-min-version="" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100734 2561 flags.go:64] FLAG: --tls-private-key-file="" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100737 2561 flags.go:64] FLAG: --topology-manager-policy="none" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100741 2561 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 27 23:36:38.108800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100744 2561 flags.go:64] FLAG: --topology-manager-scope="container" Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100747 2561 flags.go:64] FLAG: --v="2" Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100751 2561 flags.go:64] FLAG: --version="false" Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100755 2561 flags.go:64] FLAG: --vmodule="" Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100758 2561 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.100762 2561 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100871 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100876 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100879 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100882 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100884 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100888 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100891 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100893 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100896 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100899 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100901 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100904 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100906 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100909 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100911 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:36:38.109457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100915 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100918 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100921 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100924 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100927 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100929 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100932 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100934 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100937 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100940 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100942 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100945 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100947 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100950 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100952 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100955 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100958 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100961 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100963 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100966 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:36:38.109995 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100968 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100971 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100975 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100977 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100980 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100983 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100985 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100988 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100990 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100993 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100996 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.100999 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101002 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101005 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101008 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101012 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101016 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101020 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101023 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:36:38.110492 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101026 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101029 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101032 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101035 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101039 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101041 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101044 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101047 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101050 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101052 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101055 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101057 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101060 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101063 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101065 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101070 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101072 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101075 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101077 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101080 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:36:38.111054 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101083 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101085 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101088 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101090 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101093 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101095 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101099 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101101 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101104 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101106 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101109 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101112 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101115 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101117 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101120 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101122 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101125 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101128 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101130 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101133 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:36:38.111626 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101135 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101138 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101140 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101143 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.101146 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.101649 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.108338 2561 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.108353 2561 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.108992 2561 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.108999 2561 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109062 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109067 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109070 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109074 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:36:38.112224 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109077 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109079 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109082 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109085 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109087 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109090 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109093 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109095 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109104 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109106 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109109 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109112 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109114 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109117 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109119 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109122 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109126 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109129 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109132 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:36:38.112826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109134 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109137 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109140 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109143 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109146 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109148 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109152 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109154 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109157 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109160 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109162 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109165 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109167 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109170 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109172 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109175 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109177 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109180 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109182 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109185 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:36:38.113570 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109188 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109191 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109201 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109203 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109206 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109208 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109211 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109213 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109216 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109218 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109221 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109223 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109225 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109228 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109231 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109233 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109236 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109239 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109242 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:36:38.114112 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109246 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109249 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109252 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109254 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109257 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109260 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109262 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109265 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109268 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109270 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109272 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109275 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109278 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109280 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109283 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109285 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109293 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109296 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109298 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109301 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:36:38.114568 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109303 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109306 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109309 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109311 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109313 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109316 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109318 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109321 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109323 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109326 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109330 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109334 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109337 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109341 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109344 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109347 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:36:38.115062 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109349 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.109354 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109523 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109528 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109532 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109535 2561 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109537 2561 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109540 2561 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109542 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109545 2561 feature_gate.go:328] unrecognized feature gate: Example2 Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109547 2561 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109550 2561 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109552 2561 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109560 2561 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109563 2561 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 27 23:36:38.115457 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109566 2561 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109568 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109570 2561 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109573 2561 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109575 2561 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109578 2561 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109580 2561 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109584 2561 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109587 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109590 2561 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109593 2561 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109596 2561 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109599 2561 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109602 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109605 2561 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109607 2561 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109610 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109612 2561 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109615 2561 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 27 23:36:38.115848 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109618 2561 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109622 2561 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109624 2561 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109627 2561 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109629 2561 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109632 2561 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109634 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109637 2561 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109645 2561 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109648 2561 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109650 2561 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109653 2561 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109661 2561 feature_gate.go:328] unrecognized feature gate: Example Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109664 2561 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109666 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109680 2561 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109684 2561 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109687 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109690 2561 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 27 23:36:38.116321 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109693 2561 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109695 2561 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109698 2561 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109701 2561 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109704 2561 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109707 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109709 2561 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109721 2561 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109725 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109727 2561 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109730 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109733 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109735 2561 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109738 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109741 2561 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109743 2561 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109746 2561 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109748 2561 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109751 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109753 2561 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 27 23:36:38.116801 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109755 2561 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109758 2561 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109760 2561 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109763 2561 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109765 2561 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109768 2561 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109777 2561 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109780 2561 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109783 2561 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109785 2561 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109788 2561 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109790 2561 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109793 2561 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109795 2561 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109798 2561 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109800 2561 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109802 2561 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109805 2561 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109808 2561 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109810 2561 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 27 23:36:38.117271 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109813 2561 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109815 2561 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109818 2561 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109821 2561 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109823 2561 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109826 2561 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109828 2561 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:38.109831 2561 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.109836 2561 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.109936 2561 server.go:962] "Client rotation is on, will bootstrap in background" Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.113099 2561 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.114530 2561 server.go:1019] "Starting client certificate rotation" Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.114634 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.114695 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 27 23:36:38.117787 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.116618 2561 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 23:36:38.118136 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.117694 2561 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 27 23:36:38.124502 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.124480 2561 log.go:25] "Validated CRI v1 runtime API" Jul 27 23:36:38.126510 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.126495 2561 log.go:25] "Validated CRI v1 image API" Jul 27 23:36:38.128541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.128527 2561 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 27 23:36:38.130633 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.130602 2561 fs.go:135] Filesystem UUIDs: map[47aa455a-2e4e-4406-907f-c4e0e966fb85:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e9e2a961-c270-41c5-8909-f13975d8ff6d:/dev/nvme0n1p3] Jul 27 23:36:38.130741 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.130631 2561 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 27 23:36:38.135500 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.135367 2561 manager.go:216] Machine: {Timestamp:2026-07-27 23:36:38.134329267 +0000 UTC m=+0.254095516 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099509 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2059dcf0c0acc296a18b5173884c93 SystemUUID:ec2059dc-f0c0-acc2-96a1-8b5173884c93 BootID:e37c1876-8430-4288-98e1-8c65fb2f0556 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:0a:07:81:69:3d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:0a:07:81:69:3d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:35:24:d8:6b:26 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 27 23:36:38.135500 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.135492 2561 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 27 23:36:38.135632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.135590 2561 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 27 23:36:38.138246 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.138220 2561 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 27 23:36:38.138393 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.138254 2561 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-203.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 27 23:36:38.139356 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.139343 2561 topology_manager.go:138] "Creating topology manager with none policy" Jul 27 23:36:38.139356 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.139356 2561 container_manager_linux.go:309] "Creating device plugin manager" Jul 27 23:36:38.139447 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.139363 2561 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 23:36:38.139914 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.139903 2561 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 27 23:36:38.140914 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.140897 2561 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 27 23:36:38.142641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.142628 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 27 23:36:38.143185 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.143160 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 23:36:38.143777 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.143756 2561 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 27 23:36:38.143864 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.143803 2561 kubelet.go:486] "Attempting to sync node with API server" Jul 27 23:36:38.143864 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.143822 2561 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 27 23:36:38.143864 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.143842 2561 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 27 23:36:38.143864 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.143856 2561 kubelet.go:398] "Adding apiserver pod source" Jul 27 23:36:38.144034 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.143870 2561 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 27 23:36:38.144270 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.144253 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 23:36:38.144350 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.144283 2561 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 27 23:36:38.146504 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.146483 2561 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 27 23:36:38.148258 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.148245 2561 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 27 23:36:38.148302 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.148274 2561 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 27 23:36:38.149692 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149679 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149696 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149702 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149708 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149722 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149728 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149736 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149742 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 27 23:36:38.149745 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149749 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 27 23:36:38.149952 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149754 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 27 23:36:38.149952 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149762 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 27 23:36:38.149952 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.149767 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 27 23:36:38.150902 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.150888 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 27 23:36:38.150902 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.150900 2561 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 27 23:36:38.154050 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.154035 2561 server.go:1268] "Started kubelet" Jul 27 23:36:38.154552 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.154157 2561 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 27 23:36:38.154552 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.154221 2561 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 27 23:36:38.154552 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.154224 2561 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 27 23:36:38.154880 ip-10-0-142-203 systemd[1]: Started Kubernetes Kubelet. Jul 27 23:36:38.155423 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.155396 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 27 23:36:38.155423 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.155409 2561 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-203.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 27 23:36:38.155612 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.155502 2561 server.go:310] "Adding debug handlers to kubelet server" Jul 27 23:36:38.155863 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.155830 2561 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 27 23:36:38.156119 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.156034 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 27 23:36:38.159566 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.159548 2561 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 27 23:36:38.160878 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.160030 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-203.ec2.internal.18c649c199a1af28 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-203.ec2.internal,UID:ip-10-0-142-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-203.ec2.internal,},FirstTimestamp:2026-07-27 23:36:38.154006312 +0000 UTC m=+0.273772560,LastTimestamp:2026-07-27 23:36:38.154006312 +0000 UTC m=+0.273772560,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-203.ec2.internal,}" Jul 27 23:36:38.163270 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.163253 2561 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 27 23:36:38.163270 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.163262 2561 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 27 23:36:38.163406 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.163332 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:38.163840 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.163819 2561 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 27 23:36:38.163921 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.163857 2561 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 27 23:36:38.163921 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.163871 2561 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 27 23:36:38.164033 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164009 2561 reconstruct.go:97] "Volume reconstruction finished" Jul 27 23:36:38.164033 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164017 2561 reconciler.go:29] "Reconciler: start to sync state" Jul 27 23:36:38.164164 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164144 2561 factory.go:55] Registering systemd factory Jul 27 23:36:38.164164 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164167 2561 factory.go:223] Registration of the systemd container factory successfully Jul 27 23:36:38.164519 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164501 2561 factory.go:153] Registering CRI-O factory Jul 27 23:36:38.164592 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164522 2561 factory.go:223] Registration of the crio container factory successfully Jul 27 23:36:38.164592 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164571 2561 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 27 23:36:38.164717 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164596 2561 factory.go:103] Registering Raw factory Jul 27 23:36:38.164717 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.164609 2561 manager.go:1196] Started watching for new ooms in manager Jul 27 23:36:38.165223 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.165205 2561 manager.go:349] Starting recovery of all containers Jul 27 23:36:38.168838 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.168809 2561 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-203.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 27 23:36:38.168976 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.168950 2561 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 27 23:36:38.178350 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.178327 2561 manager.go:354] Recovery completed Jul 27 23:36:38.182546 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.182532 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:36:38.184271 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.184250 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:36:38.184339 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.184285 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:36:38.184339 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.184295 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:36:38.184874 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.184858 2561 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 27 23:36:38.184957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.184874 2561 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 27 23:36:38.184957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.184892 2561 state_mem.go:36] "Initialized new in-memory state store" Jul 27 23:36:38.184957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.184866 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:36:38.185574 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.185557 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:36:38.185646 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.185587 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:36:38.185646 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.185601 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:36:38.186317 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.186260 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-203.ec2.internal.18c649c19b6f7ea4 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-203.ec2.internal,UID:ip-10-0-142-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-203.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-203.ec2.internal,},FirstTimestamp:2026-07-27 23:36:38.184271524 +0000 UTC m=+0.304037772,LastTimestamp:2026-07-27 23:36:38.184271524 +0000 UTC m=+0.304037772,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-203.ec2.internal,}" Jul 27 23:36:38.186404 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.186393 2561 policy_none.go:49] "None policy: Start" Jul 27 23:36:38.186484 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.186409 2561 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 27 23:36:38.186484 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.186419 2561 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 27 23:36:38.187270 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.187259 2561 policy_none.go:47] "Start" Jul 27 23:36:38.187617 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.187602 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zr7vm" Jul 27 23:36:38.194729 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.194712 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zr7vm" Jul 27 23:36:38.196109 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.196048 2561 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-203.ec2.internal.18c649c19b6fc62f default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-203.ec2.internal,UID:ip-10-0-142-203.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-142-203.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-142-203.ec2.internal,},FirstTimestamp:2026-07-27 23:36:38.184289839 +0000 UTC m=+0.304056086,LastTimestamp:2026-07-27 23:36:38.184289839 +0000 UTC m=+0.304056086,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-203.ec2.internal,}" Jul 27 23:36:38.226038 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.225887 2561 manager.go:337] "Starting Device Plugin manager" Jul 27 23:36:38.226142 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.226067 2561 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 27 23:36:38.226142 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.226080 2561 server.go:87] "Starting device plugin registration server" Jul 27 23:36:38.226298 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.226288 2561 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 27 23:36:38.226339 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.226302 2561 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 27 23:36:38.226544 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.226393 2561 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 27 23:36:38.226544 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.226469 2561 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 27 23:36:38.226544 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.226478 2561 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 27 23:36:38.227016 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.227000 2561 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 27 23:36:38.227082 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.227047 2561 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:38.309110 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.309086 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 27 23:36:38.310169 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.310157 2561 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 27 23:36:38.310222 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.310194 2561 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 27 23:36:38.310222 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.310213 2561 kubelet.go:2448] "Starting kubelet main sync loop" Jul 27 23:36:38.310297 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.310247 2561 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 27 23:36:38.312600 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.312583 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:36:38.327430 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.327406 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:36:38.328175 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.328160 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:36:38.328253 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.328192 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:36:38.328253 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.328207 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:36:38.328253 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.328234 2561 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.339393 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.339377 2561 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.339393 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.339400 2561 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-203.ec2.internal\": node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:38.363148 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.363108 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:38.410765 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.410748 2561 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal"] Jul 27 23:36:38.410834 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.410821 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:36:38.412089 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.412066 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:36:38.412141 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.412095 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:36:38.412141 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.412106 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:36:38.413158 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413147 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:36:38.413294 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413277 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.413425 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413305 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:36:38.413815 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413801 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:36:38.413885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413828 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:36:38.413885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413801 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:36:38.413885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413866 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:36:38.413977 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413887 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:36:38.413977 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.413841 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:36:38.415046 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.415033 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.415159 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.415056 2561 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 27 23:36:38.415656 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.415640 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientMemory" Jul 27 23:36:38.415738 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.415666 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasNoDiskPressure" Jul 27 23:36:38.415738 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.415692 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeHasSufficientPID" Jul 27 23:36:38.443344 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.443326 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-203.ec2.internal\" not found" node="ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.447488 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.447475 2561 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-203.ec2.internal\" not found" node="ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.464166 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.464146 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:38.465307 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.465290 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e5721a4767c6e3cd7e44da945b03d31-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal\" (UID: \"3e5721a4767c6e3cd7e44da945b03d31\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.465379 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.465320 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e5721a4767c6e3cd7e44da945b03d31-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal\" (UID: \"3e5721a4767c6e3cd7e44da945b03d31\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.465379 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.465344 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c2a396ccbfd4c8c59eadf247f295e9f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-203.ec2.internal\" (UID: \"0c2a396ccbfd4c8c59eadf247f295e9f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.565039 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.565005 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:38.566122 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.566105 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e5721a4767c6e3cd7e44da945b03d31-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal\" (UID: \"3e5721a4767c6e3cd7e44da945b03d31\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.566182 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.566129 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e5721a4767c6e3cd7e44da945b03d31-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal\" (UID: \"3e5721a4767c6e3cd7e44da945b03d31\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.566182 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.566147 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c2a396ccbfd4c8c59eadf247f295e9f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-203.ec2.internal\" (UID: \"0c2a396ccbfd4c8c59eadf247f295e9f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.566269 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.566172 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/0c2a396ccbfd4c8c59eadf247f295e9f-config\") pod \"kube-apiserver-proxy-ip-10-0-142-203.ec2.internal\" (UID: \"0c2a396ccbfd4c8c59eadf247f295e9f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.566269 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.566217 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e5721a4767c6e3cd7e44da945b03d31-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal\" (UID: \"3e5721a4767c6e3cd7e44da945b03d31\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.566269 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:38.566215 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/3e5721a4767c6e3cd7e44da945b03d31-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal\" (UID: \"3e5721a4767c6e3cd7e44da945b03d31\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:38.665692 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:38.665614 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:40.401294 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.401276 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:40.405801 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.405777 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" Jul 27 23:36:40.422037 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:40.422016 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:40.522486 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:40.522445 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:40.622908 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:40.622845 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:40.723446 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:40.723421 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:40.769642 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.769626 2561 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 27 23:36:40.770014 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.769768 2561 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 27 23:36:40.823616 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:40.823591 2561 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-203.ec2.internal\" not found" Jul 27 23:36:40.854151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.854117 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-26 23:31:39 +0000 UTC" deadline="2028-02-10 04:58:46.362575397 +0000 UTC" Jul 27 23:36:40.854151 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.854145 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13493h22m5.508432998s" Jul 27 23:36:40.890731 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.890679 2561 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:36:40.919492 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.919474 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" Jul 27 23:36:40.928449 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.928422 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 23:36:40.929879 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.929862 2561 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" Jul 27 23:36:40.939873 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.939857 2561 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 27 23:36:40.999598 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:40.999574 2561 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:36:41.032517 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:41.032490 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c2a396ccbfd4c8c59eadf247f295e9f.slice/crio-682f52302ac70e83965966deddb18dad658bdadc9839c4cf70162fece42b380d WatchSource:0}: Error finding container 682f52302ac70e83965966deddb18dad658bdadc9839c4cf70162fece42b380d: Status 404 returned error can't find the container with id 682f52302ac70e83965966deddb18dad658bdadc9839c4cf70162fece42b380d Jul 27 23:36:41.033000 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:41.032981 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e5721a4767c6e3cd7e44da945b03d31.slice/crio-1d9741a46c73e1796b72e84a4d5cc84c3b1ba3c7f2501191d37a28a8692554df WatchSource:0}: Error finding container 1d9741a46c73e1796b72e84a4d5cc84c3b1ba3c7f2501191d37a28a8692554df: Status 404 returned error can't find the container with id 1d9741a46c73e1796b72e84a4d5cc84c3b1ba3c7f2501191d37a28a8692554df Jul 27 23:36:41.037301 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.037282 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 23:36:41.403274 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.403242 2561 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:36:41.800339 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.800307 2561 apiserver.go:52] "Watching apiserver" Jul 27 23:36:41.807764 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.807738 2561 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 27 23:36:41.808682 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.808650 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg","openshift-cluster-node-tuning-operator/tuned-rntkp","openshift-dns/node-resolver-xmnf8","openshift-image-registry/node-ca-5zbj9","openshift-multus/multus-additional-cni-plugins-bqmzs","openshift-multus/network-metrics-daemon-9b8f5","openshift-network-diagnostics/network-check-target-d4xtr","kube-system/global-pull-secret-syncer-25t97","kube-system/konnectivity-agent-xkg6r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal","openshift-multus/multus-vns6x","openshift-network-operator/iptables-alerter-bkkrs","openshift-ovn-kubernetes/ovnkube-node-7rpxv","kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal"] Jul 27 23:36:41.810265 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.810242 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.811336 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.811316 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.812619 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.812593 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.813422 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.813401 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-47kzq\"" Jul 27 23:36:41.813616 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.813595 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.813691 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.813635 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 27 23:36:41.813749 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.813680 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 27 23:36:41.813826 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.813596 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:36:41.813964 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.813891 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 27 23:36:41.814747 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.814295 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 27 23:36:41.814747 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.814577 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 27 23:36:41.814898 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.814811 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nt2r8\"" Jul 27 23:36:41.815804 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.815071 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.815804 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.815134 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-74dl6\"" Jul 27 23:36:41.815804 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.815345 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 27 23:36:41.816341 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.816094 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:41.816341 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.816156 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.816950 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.817324 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.817329 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.817546 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.817571 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-8d749\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.817710 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.817770 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.817967 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fbqzt\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.817989 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.818192 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 27 23:36:41.818304 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.818206 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 27 23:36:41.818857 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.818395 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 27 23:36:41.818857 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.818587 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.818857 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.818640 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:41.819982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.819575 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:41.819982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.819744 2561 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 27 23:36:41.821259 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.820924 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.825284 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.825262 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.825749 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.825724 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:41.827542 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.827255 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 27 23:36:41.827542 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.827497 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vt7nb\"" Jul 27 23:36:41.828067 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.827853 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c76qs\"" Jul 27 23:36:41.828067 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.827901 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 27 23:36:41.828067 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.827996 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 27 23:36:41.828271 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828114 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828660 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d81e8510-04d0-4c8e-8b56-281f0ccdd165-dbus\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828703 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5-konnectivity-ca\") pod \"konnectivity-agent-xkg6r\" (UID: \"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5\") " pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828732 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-sys-fs\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828762 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysconfig\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828791 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-tuned\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828819 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b627ead5-83e7-472c-b2ac-1b126e15fc3c-serviceca\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828848 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gns2p\" (UniqueName: \"kubernetes.io/projected/b627ead5-83e7-472c-b2ac-1b126e15fc3c-kube-api-access-gns2p\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828872 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80f5609e-89bb-49c9-b9b6-465e791cde32-tmp\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828901 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-socket-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828906 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828929 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-modprobe-d\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828957 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysctl-d\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.828984 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b627ead5-83e7-472c-b2ac-1b126e15fc3c-host\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829008 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-cnibin\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829037 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-cni-binary-copy\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829091 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-registration-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.829748 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829134 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysctl-conf\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829161 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-systemd\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829220 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-run\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829308 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-lib-modules\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829321 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-rf8cr\"" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-var-lib-kubelet\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829364 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829392 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829422 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829453 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829462 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829484 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hksv8\" (UniqueName: \"kubernetes.io/projected/c22c2085-c84d-4411-875e-4ea9a1d6686f-kube-api-access-hksv8\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829517 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-system-cni-dir\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829544 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829710 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-etc-selinux\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829763 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-host\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.830721 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829805 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t59mk\" (UniqueName: \"kubernetes.io/projected/80f5609e-89bb-49c9-b9b6-465e791cde32-kube-api-access-t59mk\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829851 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-os-release\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829887 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829929 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5-agent-certs\") pod \"konnectivity-agent-xkg6r\" (UID: \"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5\") " pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.829956 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eefe218-95cd-4c4d-b878-fc8864079d8b-tmp-dir\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3eefe218-95cd-4c4d-b878-fc8864079d8b-hosts-file\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830082 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hmmb\" (UniqueName: \"kubernetes.io/projected/a94ff868-7338-43d5-b8b0-49ebf8277545-kube-api-access-5hmmb\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830115 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-device-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830150 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-kubernetes\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830173 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-sys\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830200 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lh9lg\" (UniqueName: \"kubernetes.io/projected/3eefe218-95cd-4c4d-b878-fc8864079d8b-kube-api-access-lh9lg\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830229 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830258 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxtzc\" (UniqueName: \"kubernetes.io/projected/50cc0ec6-df88-4006-bf61-2038cd76a81c-kube-api-access-gxtzc\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830288 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d81e8510-04d0-4c8e-8b56-281f0ccdd165-kubelet-config\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830480 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830512 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830734 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830860 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 27 23:36:41.831651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.830955 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-z5ndr\"" Jul 27 23:36:41.835514 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.835490 2561 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 27 23:36:41.852394 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.852373 2561 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pv7dp" Jul 27 23:36:41.862094 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.861987 2561 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pv7dp" Jul 27 23:36:41.920566 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.920544 2561 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 27 23:36:41.931391 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931341 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.931391 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931386 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-netns\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.931563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931414 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm574\" (UniqueName: \"kubernetes.io/projected/85cbe500-4ed4-4152-bc9c-0f325febf969-kube-api-access-lm574\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:41.931563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931441 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-log-socket\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.931563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931463 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-cni-netd\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.931563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931489 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hksv8\" (UniqueName: \"kubernetes.io/projected/c22c2085-c84d-4411-875e-4ea9a1d6686f-kube-api-access-hksv8\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.931563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931513 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.931563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931538 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.931563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931563 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-kubelet\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.931892 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931585 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-host\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.931892 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931610 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/547908da-1504-41ed-b6b8-16332e8a65e6-multus-daemon-config\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.931892 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqrnm\" (UniqueName: \"kubernetes.io/projected/547908da-1504-41ed-b6b8-16332e8a65e6-kube-api-access-qqrnm\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.931892 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.931653 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:41.932103 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.932029 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret podName:d81e8510-04d0-4c8e-8b56-281f0ccdd165 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:42.431695017 +0000 UTC m=+2.896077151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret") pod "global-pull-secret-syncer-25t97" (UID: "d81e8510-04d0-4c8e-8b56-281f0ccdd165") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:41.932103 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932033 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.932219 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.931653 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-cni-bin\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932272 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932242 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-ovnkube-config\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932330 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932270 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-ovnkube-script-lib\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932330 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932295 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r5kf\" (UniqueName: \"kubernetes.io/projected/5b97275d-8b9b-450a-9be4-5255067ffb36-kube-api-access-6r5kf\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932461 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932338 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hmmb\" (UniqueName: \"kubernetes.io/projected/a94ff868-7338-43d5-b8b0-49ebf8277545-kube-api-access-5hmmb\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:41.932461 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932364 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-device-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.932461 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-sys\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.932461 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lh9lg\" (UniqueName: \"kubernetes.io/projected/3eefe218-95cd-4c4d-b878-fc8864079d8b-kube-api-access-lh9lg\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.932461 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932440 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-k8s-cni-cncf-io\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932467 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-kubelet\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932490 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-node-log\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932514 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932540 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-socket-dir-parent\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932564 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-ovn\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932589 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5-konnectivity-ca\") pod \"konnectivity-agent-xkg6r\" (UID: \"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5\") " pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932589 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-multus-certs\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932632 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-slash\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932665 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-etc-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932676 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-host\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.932695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932687 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-env-overrides\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-sys-fs\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932735 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-tuned\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932758 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b627ead5-83e7-472c-b2ac-1b126e15fc3c-serviceca\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932782 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-etc-kubernetes\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932804 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b627ead5-83e7-472c-b2ac-1b126e15fc3c-host\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-cni-binary-copy\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932849 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-hostroot\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932899 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-registration-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932921 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-run\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932969 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-device-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.932998 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-var-lib-kubelet\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933020 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933046 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-cni-multus\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933069 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-run\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933087 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85cbe500-4ed4-4152-bc9c-0f325febf969-host-slash\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933151 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-sys\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.933279 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.933170 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.933216 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs podName:a94ff868-7338-43d5-b8b0-49ebf8277545 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:42.433200318 +0000 UTC m=+2.897582437 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs") pod "network-metrics-daemon-9b8f5" (UID: "a94ff868-7338-43d5-b8b0-49ebf8277545") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933367 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-var-lib-kubelet\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933493 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-registration-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933501 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-sys-fs\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933530 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933561 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933586 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-system-cni-dir\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933611 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-system-cni-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933626 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-cni-bin\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933649 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-run-netns\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933666 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-etc-selinux\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933682 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t59mk\" (UniqueName: \"kubernetes.io/projected/80f5609e-89bb-49c9-b9b6-465e791cde32-kube-api-access-t59mk\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933696 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-os-release\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933737 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933754 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/b627ead5-83e7-472c-b2ac-1b126e15fc3c-serviceca\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.934134 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933761 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5-agent-certs\") pod \"konnectivity-agent-xkg6r\" (UID: \"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5\") " pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933754 2561 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933777 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5-konnectivity-ca\") pod \"konnectivity-agent-xkg6r\" (UID: \"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5\") " pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933780 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b97275d-8b9b-450a-9be4-5255067ffb36-ovn-node-metrics-cert\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933819 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eefe218-95cd-4c4d-b878-fc8864079d8b-tmp-dir\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933826 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-os-release\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933840 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3eefe218-95cd-4c4d-b878-fc8864079d8b-hosts-file\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933853 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-system-cni-dir\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933862 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-kubernetes\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933912 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-etc-selinux\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933937 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxtzc\" (UniqueName: \"kubernetes.io/projected/50cc0ec6-df88-4006-bf61-2038cd76a81c-kube-api-access-gxtzc\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933956 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-os-release\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933973 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-systemd\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933993 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d81e8510-04d0-4c8e-8b56-281f0ccdd165-kubelet-config\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.933989 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-cni-binary-copy\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934272 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b627ead5-83e7-472c-b2ac-1b126e15fc3c-host\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934347 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/50cc0ec6-df88-4006-bf61-2038cd76a81c-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.934836 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-systemd-units\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934442 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3eefe218-95cd-4c4d-b878-fc8864079d8b-tmp-dir\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934449 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d81e8510-04d0-4c8e-8b56-281f0ccdd165-dbus\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934459 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/d81e8510-04d0-4c8e-8b56-281f0ccdd165-kubelet-config\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934481 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-cni-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934505 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3eefe218-95cd-4c4d-b878-fc8864079d8b-hosts-file\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934560 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-kubernetes\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934591 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-cnibin\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysconfig\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934659 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gns2p\" (UniqueName: \"kubernetes.io/projected/b627ead5-83e7-472c-b2ac-1b126e15fc3c-kube-api-access-gns2p\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934663 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysconfig\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934696 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/d81e8510-04d0-4c8e-8b56-281f0ccdd165-dbus\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934774 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/547908da-1504-41ed-b6b8-16332e8a65e6-cni-binary-copy\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934830 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934899 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80f5609e-89bb-49c9-b9b6-465e791cde32-tmp\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934937 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-socket-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.935581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.934980 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-modprobe-d\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935044 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysctl-d\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935099 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-cnibin\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935117 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c22c2085-c84d-4411-875e-4ea9a1d6686f-socket-dir\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935121 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-modprobe-d\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935135 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-conf-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935161 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85cbe500-4ed4-4152-bc9c-0f325febf969-iptables-alerter-script\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935166 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/50cc0ec6-df88-4006-bf61-2038cd76a81c-cnibin\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935202 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-var-lib-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935247 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysctl-conf\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935252 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysctl-d\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935281 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-systemd\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935308 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-lib-modules\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935362 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-sysctl-conf\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935412 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-systemd\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.936383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.935460 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/80f5609e-89bb-49c9-b9b6-465e791cde32-lib-modules\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.940537 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.940458 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5-agent-certs\") pod \"konnectivity-agent-xkg6r\" (UID: \"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5\") " pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:41.940651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.940617 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/80f5609e-89bb-49c9-b9b6-465e791cde32-etc-tuned\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.942583 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.942562 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:36:41.942667 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.942588 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:36:41.942667 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.942601 2561 projected.go:196] Error preparing data for projected volume kube-api-access-b4sfb for pod openshift-network-diagnostics/network-check-target-d4xtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:41.942667 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:41.942654 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb podName:fac2160a-9b05-4bfc-8e98-6205cbf4d252 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:42.442638402 +0000 UTC m=+2.907020532 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-b4sfb" (UniqueName: "kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb") pod "network-check-target-d4xtr" (UID: "fac2160a-9b05-4bfc-8e98-6205cbf4d252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:41.942839 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.942716 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hksv8\" (UniqueName: \"kubernetes.io/projected/c22c2085-c84d-4411-875e-4ea9a1d6686f-kube-api-access-hksv8\") pod \"aws-ebs-csi-driver-node-ggpmg\" (UID: \"c22c2085-c84d-4411-875e-4ea9a1d6686f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:41.943368 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.943341 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/80f5609e-89bb-49c9-b9b6-465e791cde32-tmp\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.945519 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.945471 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hmmb\" (UniqueName: \"kubernetes.io/projected/a94ff868-7338-43d5-b8b0-49ebf8277545-kube-api-access-5hmmb\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:41.946218 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.946195 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gns2p\" (UniqueName: \"kubernetes.io/projected/b627ead5-83e7-472c-b2ac-1b126e15fc3c-kube-api-access-gns2p\") pod \"node-ca-5zbj9\" (UID: \"b627ead5-83e7-472c-b2ac-1b126e15fc3c\") " pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:41.946627 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.946604 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t59mk\" (UniqueName: \"kubernetes.io/projected/80f5609e-89bb-49c9-b9b6-465e791cde32-kube-api-access-t59mk\") pod \"tuned-rntkp\" (UID: \"80f5609e-89bb-49c9-b9b6-465e791cde32\") " pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:41.946835 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.946766 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lh9lg\" (UniqueName: \"kubernetes.io/projected/3eefe218-95cd-4c4d-b878-fc8864079d8b-kube-api-access-lh9lg\") pod \"node-resolver-xmnf8\" (UID: \"3eefe218-95cd-4c4d-b878-fc8864079d8b\") " pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:41.947668 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.947646 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxtzc\" (UniqueName: \"kubernetes.io/projected/50cc0ec6-df88-4006-bf61-2038cd76a81c-kube-api-access-gxtzc\") pod \"multus-additional-cni-plugins-bqmzs\" (UID: \"50cc0ec6-df88-4006-bf61-2038cd76a81c\") " pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:41.971183 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.971136 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" event={"ID":"0c2a396ccbfd4c8c59eadf247f295e9f","Type":"ContainerStarted","Data":"682f52302ac70e83965966deddb18dad658bdadc9839c4cf70162fece42b380d"} Jul 27 23:36:41.973467 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:41.973443 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" event={"ID":"3e5721a4767c6e3cd7e44da945b03d31","Type":"ContainerStarted","Data":"1d9741a46c73e1796b72e84a4d5cc84c3b1ba3c7f2501191d37a28a8692554df"} Jul 27 23:36:42.036395 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036364 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-socket-dir-parent\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036407 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-ovn\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036434 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-multus-certs\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036457 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-slash\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036481 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-etc-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-env-overrides\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036511 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-socket-dir-parent\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036541 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036535 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-etc-kubernetes\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036545 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-slash\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036561 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-hostroot\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036567 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-etc-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036585 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036611 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-etc-kubernetes\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036615 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-cni-multus\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036511 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-ovn\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036659 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-hostroot\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036680 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036699 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-cni-multus\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036587 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-multus-certs\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036713 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85cbe500-4ed4-4152-bc9c-0f325febf969-host-slash\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036746 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/85cbe500-4ed4-4152-bc9c-0f325febf969-host-slash\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036752 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-system-cni-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-cni-bin\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036827 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-run-netns\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036835 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-system-cni-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.036885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036886 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b97275d-8b9b-450a-9be4-5255067ffb36-ovn-node-metrics-cert\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036914 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-run-netns\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036917 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-os-release\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036887 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-cni-bin\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036952 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-systemd\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036968 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-os-release\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.036978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-systemd-units\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037002 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-systemd\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037003 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-cni-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037033 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-systemd-units\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-cnibin\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-cni-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037106 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/547908da-1504-41ed-b6b8-16332e8a65e6-cni-binary-copy\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037112 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-env-overrides\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037116 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-cnibin\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037132 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037158 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037186 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-conf-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.037838 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037207 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85cbe500-4ed4-4152-bc9c-0f325febf969-iptables-alerter-script\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037231 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-var-lib-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037238 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-run-ovn-kubernetes\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037292 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-netns\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037297 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-multus-conf-dir\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037318 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm574\" (UniqueName: \"kubernetes.io/projected/85cbe500-4ed4-4152-bc9c-0f325febf969-kube-api-access-lm574\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037343 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-log-socket\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037366 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-cni-netd\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037393 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-kubelet\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037411 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-netns\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/547908da-1504-41ed-b6b8-16332e8a65e6-multus-daemon-config\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037440 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqrnm\" (UniqueName: \"kubernetes.io/projected/547908da-1504-41ed-b6b8-16332e8a65e6-kube-api-access-qqrnm\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037457 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-run-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037462 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-cni-bin\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037295 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-var-lib-openvswitch\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037484 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-ovnkube-config\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037509 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-kubelet\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.038702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037517 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-ovnkube-script-lib\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037544 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r5kf\" (UniqueName: \"kubernetes.io/projected/5b97275d-8b9b-450a-9be4-5255067ffb36-kube-api-access-6r5kf\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037572 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-k8s-cni-cncf-io\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037594 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-kubelet\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037618 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-node-log\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037674 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-node-log\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037685 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-log-socket\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037520 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-cni-bin\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037735 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-var-lib-kubelet\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037787 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/85cbe500-4ed4-4152-bc9c-0f325febf969-iptables-alerter-script\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037944 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5b97275d-8b9b-450a-9be4-5255067ffb36-host-cni-netd\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037947 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-ovnkube-config\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.037982 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/547908da-1504-41ed-b6b8-16332e8a65e6-host-run-k8s-cni-cncf-io\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.038218 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/547908da-1504-41ed-b6b8-16332e8a65e6-multus-daemon-config\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.038227 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/547908da-1504-41ed-b6b8-16332e8a65e6-cni-binary-copy\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.039572 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.038322 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5b97275d-8b9b-450a-9be4-5255067ffb36-ovnkube-script-lib\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.040308 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.040036 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5b97275d-8b9b-450a-9be4-5255067ffb36-ovn-node-metrics-cert\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.045346 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.045318 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqrnm\" (UniqueName: \"kubernetes.io/projected/547908da-1504-41ed-b6b8-16332e8a65e6-kube-api-access-qqrnm\") pod \"multus-vns6x\" (UID: \"547908da-1504-41ed-b6b8-16332e8a65e6\") " pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.045735 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.045709 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm574\" (UniqueName: \"kubernetes.io/projected/85cbe500-4ed4-4152-bc9c-0f325febf969-kube-api-access-lm574\") pod \"iptables-alerter-bkkrs\" (UID: \"85cbe500-4ed4-4152-bc9c-0f325febf969\") " pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:42.046301 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.046281 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r5kf\" (UniqueName: \"kubernetes.io/projected/5b97275d-8b9b-450a-9be4-5255067ffb36-kube-api-access-6r5kf\") pod \"ovnkube-node-7rpxv\" (UID: \"5b97275d-8b9b-450a-9be4-5255067ffb36\") " pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.128398 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.128328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" Jul 27 23:36:42.139313 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.139282 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rntkp" Jul 27 23:36:42.148798 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.148777 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-xmnf8" Jul 27 23:36:42.155377 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.155352 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5zbj9" Jul 27 23:36:42.165226 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.165207 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" Jul 27 23:36:42.173802 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.173780 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:36:42.182802 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.182783 2561 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 27 23:36:42.182902 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.182803 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vns6x" Jul 27 23:36:42.191833 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.191817 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:36:42.199673 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.199654 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-bkkrs" Jul 27 23:36:42.441276 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.441202 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:42.441276 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.441250 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:42.441481 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.441376 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:42.441481 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.441391 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:42.441481 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.441436 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs podName:a94ff868-7338-43d5-b8b0-49ebf8277545 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:43.441420489 +0000 UTC m=+3.905802626 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs") pod "network-metrics-daemon-9b8f5" (UID: "a94ff868-7338-43d5-b8b0-49ebf8277545") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:42.441481 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.441468 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret podName:d81e8510-04d0-4c8e-8b56-281f0ccdd165 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:43.44144811 +0000 UTC m=+3.905830227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret") pod "global-pull-secret-syncer-25t97" (UID: "d81e8510-04d0-4c8e-8b56-281f0ccdd165") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:42.542621 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.542583 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:42.542787 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.542731 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:36:42.542787 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.542752 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:36:42.542787 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.542765 2561 projected.go:196] Error preparing data for projected volume kube-api-access-b4sfb for pod openshift-network-diagnostics/network-check-target-d4xtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:42.542920 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:42.542826 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb podName:fac2160a-9b05-4bfc-8e98-6205cbf4d252 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:43.542808639 +0000 UTC m=+4.007190760 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4sfb" (UniqueName: "kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb") pod "network-check-target-d4xtr" (UID: "fac2160a-9b05-4bfc-8e98-6205cbf4d252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:42.864742 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.864702 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 23:31:41 +0000 UTC" deadline="2028-05-12 22:40:28.796902576 +0000 UTC" Jul 27 23:36:42.864742 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:42.864734 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15719h3m45.932172716s" Jul 27 23:36:43.043106 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.043078 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cc0ec6_df88_4006_bf61_2038cd76a81c.slice/crio-9e691c57a54286d8d20eabead37be203f1669b80bfc780a8a18ffa381ae3bbd2 WatchSource:0}: Error finding container 9e691c57a54286d8d20eabead37be203f1669b80bfc780a8a18ffa381ae3bbd2: Status 404 returned error can't find the container with id 9e691c57a54286d8d20eabead37be203f1669b80bfc780a8a18ffa381ae3bbd2 Jul 27 23:36:43.044232 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.044207 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80f5609e_89bb_49c9_b9b6_465e791cde32.slice/crio-cc5e9a0c404057f2aba41c970fcc24b2f55b2b4bcb310ccf90905f7d5532d336 WatchSource:0}: Error finding container cc5e9a0c404057f2aba41c970fcc24b2f55b2b4bcb310ccf90905f7d5532d336: Status 404 returned error can't find the container with id cc5e9a0c404057f2aba41c970fcc24b2f55b2b4bcb310ccf90905f7d5532d336 Jul 27 23:36:43.045597 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.045543 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3eefe218_95cd_4c4d_b878_fc8864079d8b.slice/crio-5561ec6b26e63c738f3314fea228e290d612e585b80a86d0f6f2fdee5f36cf87 WatchSource:0}: Error finding container 5561ec6b26e63c738f3314fea228e290d612e585b80a86d0f6f2fdee5f36cf87: Status 404 returned error can't find the container with id 5561ec6b26e63c738f3314fea228e290d612e585b80a86d0f6f2fdee5f36cf87 Jul 27 23:36:43.049503 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.049471 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b97275d_8b9b_450a_9be4_5255067ffb36.slice/crio-a88805eda83e6fed8d6a2946fca3141ed8ff67764a78ae3729524bad06955dfd WatchSource:0}: Error finding container a88805eda83e6fed8d6a2946fca3141ed8ff67764a78ae3729524bad06955dfd: Status 404 returned error can't find the container with id a88805eda83e6fed8d6a2946fca3141ed8ff67764a78ae3729524bad06955dfd Jul 27 23:36:43.049804 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.049748 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc22c2085_c84d_4411_875e_4ea9a1d6686f.slice/crio-d4b24bd5b5b468df19cb82a9430d14c6d0da0b3340b6de1b194b6ac12f8e5303 WatchSource:0}: Error finding container d4b24bd5b5b468df19cb82a9430d14c6d0da0b3340b6de1b194b6ac12f8e5303: Status 404 returned error can't find the container with id d4b24bd5b5b468df19cb82a9430d14c6d0da0b3340b6de1b194b6ac12f8e5303 Jul 27 23:36:43.051017 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.050941 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85cbe500_4ed4_4152_bc9c_0f325febf969.slice/crio-818a3a45f09a00820746f839490bc5b8443edd4ea346b0be0fd079e35e648b09 WatchSource:0}: Error finding container 818a3a45f09a00820746f839490bc5b8443edd4ea346b0be0fd079e35e648b09: Status 404 returned error can't find the container with id 818a3a45f09a00820746f839490bc5b8443edd4ea346b0be0fd079e35e648b09 Jul 27 23:36:43.052424 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.051938 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c122ef5_5ba8_4905_8765_e5a6e3c9e2f5.slice/crio-ce15ea3a80ccbd785d7cb005c8269c3ca59ee51e4d5f6dbe3eb6de805cf56de0 WatchSource:0}: Error finding container ce15ea3a80ccbd785d7cb005c8269c3ca59ee51e4d5f6dbe3eb6de805cf56de0: Status 404 returned error can't find the container with id ce15ea3a80ccbd785d7cb005c8269c3ca59ee51e4d5f6dbe3eb6de805cf56de0 Jul 27 23:36:43.053225 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:36:43.053111 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod547908da_1504_41ed_b6b8_16332e8a65e6.slice/crio-03194df2afe06f24af401fe6f9696e1fc159cae6ead6ea4561e62fd4c3781d1c WatchSource:0}: Error finding container 03194df2afe06f24af401fe6f9696e1fc159cae6ead6ea4561e62fd4c3781d1c: Status 404 returned error can't find the container with id 03194df2afe06f24af401fe6f9696e1fc159cae6ead6ea4561e62fd4c3781d1c Jul 27 23:36:43.449306 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.449043 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:43.449306 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.449298 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:43.449479 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.449196 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:43.449479 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.449382 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs podName:a94ff868-7338-43d5-b8b0-49ebf8277545 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:45.449363725 +0000 UTC m=+5.913745844 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs") pod "network-metrics-daemon-9b8f5" (UID: "a94ff868-7338-43d5-b8b0-49ebf8277545") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:43.449479 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.449383 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:43.449479 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.449423 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret podName:d81e8510-04d0-4c8e-8b56-281f0ccdd165 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:45.449411254 +0000 UTC m=+5.913793371 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret") pod "global-pull-secret-syncer-25t97" (UID: "d81e8510-04d0-4c8e-8b56-281f0ccdd165") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:43.549928 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.549888 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:43.550138 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.550101 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:36:43.550138 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.550126 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:36:43.550248 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.550141 2561 projected.go:196] Error preparing data for projected volume kube-api-access-b4sfb for pod openshift-network-diagnostics/network-check-target-d4xtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:43.550248 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.550204 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb podName:fac2160a-9b05-4bfc-8e98-6205cbf4d252 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:45.550185379 +0000 UTC m=+6.014567500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4sfb" (UniqueName: "kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb") pod "network-check-target-d4xtr" (UID: "fac2160a-9b05-4bfc-8e98-6205cbf4d252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:43.865652 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.865585 2561 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-26 23:31:41 +0000 UTC" deadline="2027-12-28 15:36:12.375728295 +0000 UTC" Jul 27 23:36:43.865652 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.865648 2561 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12447h59m28.510083857s" Jul 27 23:36:43.969112 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.969073 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:43.969276 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.969206 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:43.969602 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.969582 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:43.969695 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.969676 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:43.969759 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.969742 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:43.969837 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:43.969821 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:43.988903 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:43.988872 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vns6x" event={"ID":"547908da-1504-41ed-b6b8-16332e8a65e6","Type":"ContainerStarted","Data":"03194df2afe06f24af401fe6f9696e1fc159cae6ead6ea4561e62fd4c3781d1c"} Jul 27 23:36:44.000158 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.000127 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bkkrs" event={"ID":"85cbe500-4ed4-4152-bc9c-0f325febf969","Type":"ContainerStarted","Data":"818a3a45f09a00820746f839490bc5b8443edd4ea346b0be0fd079e35e648b09"} Jul 27 23:36:44.010197 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.010153 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" event={"ID":"c22c2085-c84d-4411-875e-4ea9a1d6686f","Type":"ContainerStarted","Data":"d4b24bd5b5b468df19cb82a9430d14c6d0da0b3340b6de1b194b6ac12f8e5303"} Jul 27 23:36:44.013591 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.013564 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerStarted","Data":"9e691c57a54286d8d20eabead37be203f1669b80bfc780a8a18ffa381ae3bbd2"} Jul 27 23:36:44.020185 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.020158 2561 generic.go:356] "Generic (PLEG): container finished" podID="3e5721a4767c6e3cd7e44da945b03d31" containerID="8f63ab3660473a2da6e5f23929738141afabe66b01508a56f9739343fbadb81c" exitCode=0 Jul 27 23:36:44.020603 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.020287 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" event={"ID":"3e5721a4767c6e3cd7e44da945b03d31","Type":"ContainerDied","Data":"8f63ab3660473a2da6e5f23929738141afabe66b01508a56f9739343fbadb81c"} Jul 27 23:36:44.037758 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.037735 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" event={"ID":"0c2a396ccbfd4c8c59eadf247f295e9f","Type":"ContainerStarted","Data":"3288ecd59a708b64ee510f47549bfeeb2c2a43870be87370ff7eb487cdef8cec"} Jul 27 23:36:44.047931 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.047907 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkg6r" event={"ID":"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5","Type":"ContainerStarted","Data":"ce15ea3a80ccbd785d7cb005c8269c3ca59ee51e4d5f6dbe3eb6de805cf56de0"} Jul 27 23:36:44.054417 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.054371 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-203.ec2.internal" podStartSLOduration=4.054354357 podStartE2EDuration="4.054354357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:36:44.053979721 +0000 UTC m=+4.518361860" watchObservedRunningTime="2026-07-27 23:36:44.054354357 +0000 UTC m=+4.518736497" Jul 27 23:36:44.054527 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.054411 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"a88805eda83e6fed8d6a2946fca3141ed8ff67764a78ae3729524bad06955dfd"} Jul 27 23:36:44.081513 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.081483 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xmnf8" event={"ID":"3eefe218-95cd-4c4d-b878-fc8864079d8b","Type":"ContainerStarted","Data":"5561ec6b26e63c738f3314fea228e290d612e585b80a86d0f6f2fdee5f36cf87"} Jul 27 23:36:44.101942 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.101911 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rntkp" event={"ID":"80f5609e-89bb-49c9-b9b6-465e791cde32","Type":"ContainerStarted","Data":"cc5e9a0c404057f2aba41c970fcc24b2f55b2b4bcb310ccf90905f7d5532d336"} Jul 27 23:36:44.109350 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:44.109322 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5zbj9" event={"ID":"b627ead5-83e7-472c-b2ac-1b126e15fc3c","Type":"ContainerStarted","Data":"d6230f7b20962a596d1611b623683123e2ed43d7f7d91729f306dbaf14a3d420"} Jul 27 23:36:45.135649 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:45.135609 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" event={"ID":"3e5721a4767c6e3cd7e44da945b03d31","Type":"ContainerStarted","Data":"64358b0a6309e63824cdf623e170a72a16d51acb155da69ce7d882c0d139487d"} Jul 27 23:36:45.470524 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:45.470444 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:45.470524 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:45.470501 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:45.470748 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.470624 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:45.470748 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.470684 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs podName:a94ff868-7338-43d5-b8b0-49ebf8277545 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:49.470666134 +0000 UTC m=+9.935048263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs") pod "network-metrics-daemon-9b8f5" (UID: "a94ff868-7338-43d5-b8b0-49ebf8277545") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:45.471134 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.471115 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:45.471232 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.471184 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret podName:d81e8510-04d0-4c8e-8b56-281f0ccdd165 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:49.471167554 +0000 UTC m=+9.935549673 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret") pod "global-pull-secret-syncer-25t97" (UID: "d81e8510-04d0-4c8e-8b56-281f0ccdd165") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:45.571664 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:45.571629 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:45.571846 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.571822 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:36:45.571846 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.571843 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:36:45.571948 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.571859 2561 projected.go:196] Error preparing data for projected volume kube-api-access-b4sfb for pod openshift-network-diagnostics/network-check-target-d4xtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:45.571948 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.571939 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb podName:fac2160a-9b05-4bfc-8e98-6205cbf4d252 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:49.57189506 +0000 UTC m=+10.036277178 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4sfb" (UniqueName: "kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb") pod "network-check-target-d4xtr" (UID: "fac2160a-9b05-4bfc-8e98-6205cbf4d252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:45.969068 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:45.969022 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:45.969245 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.969171 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:45.969598 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:45.969579 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:45.969700 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.969675 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:45.969766 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:45.969749 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:45.969839 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:45.969820 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:47.966360 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:47.966328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:47.966783 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:47.966328 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:47.966783 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:47.966467 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:47.966783 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:47.966535 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:47.966783 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:47.966630 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:47.966783 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:47.966696 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:49.509641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:49.509477 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:49.509641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:49.509552 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:49.510169 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.509654 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:49.510169 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.509720 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret podName:d81e8510-04d0-4c8e-8b56-281f0ccdd165 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:57.509701313 +0000 UTC m=+17.974083434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret") pod "global-pull-secret-syncer-25t97" (UID: "d81e8510-04d0-4c8e-8b56-281f0ccdd165") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:49.510169 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.509720 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:49.510169 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.509783 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs podName:a94ff868-7338-43d5-b8b0-49ebf8277545 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:57.509769283 +0000 UTC m=+17.974151398 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs") pod "network-metrics-daemon-9b8f5" (UID: "a94ff868-7338-43d5-b8b0-49ebf8277545") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:49.610321 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:49.610283 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:49.610488 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.610466 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:36:49.610570 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.610499 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:36:49.610570 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.610512 2561 projected.go:196] Error preparing data for projected volume kube-api-access-b4sfb for pod openshift-network-diagnostics/network-check-target-d4xtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:49.610661 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.610578 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb podName:fac2160a-9b05-4bfc-8e98-6205cbf4d252 nodeName:}" failed. No retries permitted until 2026-07-27 23:36:57.610556138 +0000 UTC m=+18.074938270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4sfb" (UniqueName: "kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb") pod "network-check-target-d4xtr" (UID: "fac2160a-9b05-4bfc-8e98-6205cbf4d252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:49.967348 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:49.967130 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:49.967348 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.967240 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:49.967952 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:49.967913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:49.968374 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.968088 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:49.968374 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:49.968177 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:49.968374 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:49.968315 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:51.966290 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:51.966108 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:51.966290 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:51.966242 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:51.966801 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:51.966343 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:51.966801 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:51.966446 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:51.966801 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:51.966482 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:51.966801 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:51.966562 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:53.966112 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:53.966080 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:53.966580 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:53.966126 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:53.966580 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:53.966081 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:53.966580 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:53.966219 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:53.966580 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:53.966364 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:53.966580 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:53.966452 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:55.966075 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:55.966023 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:55.966492 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:55.966023 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:55.966492 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:55.966173 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:55.966492 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:55.966031 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:55.966492 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:55.966258 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:55.966492 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:55.966308 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:57.571723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:57.571682 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:57.571723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:57.571733 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:57.572231 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.571838 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:57.572231 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.571845 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:57.572231 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.571913 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret podName:d81e8510-04d0-4c8e-8b56-281f0ccdd165 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:13.571891995 +0000 UTC m=+34.036274114 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret") pod "global-pull-secret-syncer-25t97" (UID: "d81e8510-04d0-4c8e-8b56-281f0ccdd165") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:36:57.572231 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.571936 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs podName:a94ff868-7338-43d5-b8b0-49ebf8277545 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:13.571922906 +0000 UTC m=+34.036305022 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs") pod "network-metrics-daemon-9b8f5" (UID: "a94ff868-7338-43d5-b8b0-49ebf8277545") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:36:57.672946 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:57.672911 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:57.673134 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.673077 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:36:57.673134 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.673098 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:36:57.673134 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.673112 2561 projected.go:196] Error preparing data for projected volume kube-api-access-b4sfb for pod openshift-network-diagnostics/network-check-target-d4xtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:57.673314 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.673175 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb podName:fac2160a-9b05-4bfc-8e98-6205cbf4d252 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:13.673155361 +0000 UTC m=+34.137537478 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4sfb" (UniqueName: "kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb") pod "network-check-target-d4xtr" (UID: "fac2160a-9b05-4bfc-8e98-6205cbf4d252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:36:57.966875 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:57.966788 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:57.966875 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:57.966833 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:57.967105 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.966951 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:57.967105 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.967014 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:57.967105 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:57.967046 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:57.967258 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:57.967150 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:36:59.967449 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:59.967415 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:36:59.967910 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:59.967519 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:36:59.967910 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:59.967561 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:36:59.967910 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:36:59.967600 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:36:59.967910 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:59.967717 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:36:59.967910 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:36:59.967820 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:01.966989 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:01.966568 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:01.967658 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:01.966568 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:01.967658 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:01.967119 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:37:01.967658 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:01.967183 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:01.967658 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:01.966617 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:01.967658 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:01.967284 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:37:02.168212 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.168176 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vns6x" event={"ID":"547908da-1504-41ed-b6b8-16332e8a65e6","Type":"ContainerStarted","Data":"d30286e0cb5493132cef5564e712de368d9772e5faed07794987bd16ba1ec568"} Jul 27 23:37:02.169804 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.169769 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" event={"ID":"c22c2085-c84d-4411-875e-4ea9a1d6686f","Type":"ContainerStarted","Data":"576180c24ecc029ebd1a987f6fdafbacc9bf38490ecd4d744a71d3c185ac5298"} Jul 27 23:37:02.171784 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.171710 2561 generic.go:356] "Generic (PLEG): container finished" podID="50cc0ec6-df88-4006-bf61-2038cd76a81c" containerID="a6cc9f1dd333c6bc699d05134aeeca2eed32ea856d81544780f1636ae967ab0f" exitCode=0 Jul 27 23:37:02.171897 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.171856 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerDied","Data":"a6cc9f1dd333c6bc699d05134aeeca2eed32ea856d81544780f1636ae967ab0f"} Jul 27 23:37:02.176181 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.176146 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-xkg6r" event={"ID":"6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5","Type":"ContainerStarted","Data":"695dd460a70db11a419e394d42c4fc41ef9acc2e9105f05f54747aa1e09ad1d8"} Jul 27 23:37:02.179812 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.179792 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"adc2ea040eccec68e73fdea1b810e1168bd3682cd72a186797f819e49052ffbb"} Jul 27 23:37:02.179919 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.179852 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"6d2ffb77343f8955379c55890e5c7c3325d709b3ff1917c2cad1839aa343a284"} Jul 27 23:37:02.179919 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.179888 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"e1defcfb5e5d5e533077cbf6ba475c9759addcd745efac6fb16346c86f79fb77"} Jul 27 23:37:02.180046 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.179919 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"633b4e5892c6ee54c1482c43d8e83cb157548fa7d183dd8708c3c55c12656f42"} Jul 27 23:37:02.180046 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.179951 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"0e4b1c910adc291d82c37572d8b25a4ebbd3fcd8b387d6d81f793c77a766ebc4"} Jul 27 23:37:02.180046 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.179994 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"31078f8b5c961c4e027bf38cd66358f3e00faa5150af4b58e2de2341f918414c"} Jul 27 23:37:02.181495 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.181469 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-xmnf8" event={"ID":"3eefe218-95cd-4c4d-b878-fc8864079d8b","Type":"ContainerStarted","Data":"d3f2f362581e10a8b7cdfa865161b68edf0a430f25193efd95e3dd1b675f1ea1"} Jul 27 23:37:02.183152 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.183132 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rntkp" event={"ID":"80f5609e-89bb-49c9-b9b6-465e791cde32","Type":"ContainerStarted","Data":"40f85274c66906e49ee99dca602b93b41c7ace674a3fe30f610e6d23360e2235"} Jul 27 23:37:02.184589 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.184552 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-203.ec2.internal" podStartSLOduration=22.184540712 podStartE2EDuration="22.184540712s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:36:45.15098834 +0000 UTC m=+5.615370478" watchObservedRunningTime="2026-07-27 23:37:02.184540712 +0000 UTC m=+22.648922851" Jul 27 23:37:02.184701 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.184605 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5zbj9" event={"ID":"b627ead5-83e7-472c-b2ac-1b126e15fc3c","Type":"ContainerStarted","Data":"fd3f2c1402fdd8b8ddbe7206b9887a37b0e65efd0b97c33c9ae3ea45b29e442a"} Jul 27 23:37:02.184701 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.184635 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vns6x" podStartSLOduration=4.032561571 podStartE2EDuration="22.184627182s" totalImagesPullingTime="18.152065611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:02.184360981 +0000 UTC m=+22.648743131" watchObservedRunningTime="2026-07-27 23:37:02.184627182 +0000 UTC m=+22.649009325" Jul 27 23:37:02.201832 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.201757 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rntkp" podStartSLOduration=4.072702922 podStartE2EDuration="22.201741927s" totalImagesPullingTime="18.129039005s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:02.201329714 +0000 UTC m=+22.665711856" watchObservedRunningTime="2026-07-27 23:37:02.201741927 +0000 UTC m=+22.666124066" Jul 27 23:37:02.216904 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.216866 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-xmnf8" podStartSLOduration=4.105267465 podStartE2EDuration="22.216854543s" totalImagesPullingTime="18.111587078s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:02.216333982 +0000 UTC m=+22.680716122" watchObservedRunningTime="2026-07-27 23:37:02.216854543 +0000 UTC m=+22.681236681" Jul 27 23:37:02.261625 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.261590 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-xkg6r" podStartSLOduration=4.188767735 podStartE2EDuration="22.261573103s" totalImagesPullingTime="18.072805368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:02.261398296 +0000 UTC m=+22.725780435" watchObservedRunningTime="2026-07-27 23:37:02.261573103 +0000 UTC m=+22.725955241" Jul 27 23:37:02.274451 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.274416 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5zbj9" podStartSLOduration=4.179067793 podStartE2EDuration="22.274404953s" totalImagesPullingTime="18.09533716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:02.274108189 +0000 UTC m=+22.738490327" watchObservedRunningTime="2026-07-27 23:37:02.274404953 +0000 UTC m=+22.738787091" Jul 27 23:37:02.675364 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.675339 2561 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 27 23:37:02.894259 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.894079 2561 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-27T23:37:02.675359345Z","UUID":"f21fb9cf-9f1a-44ed-91fb-04fa4bfaffeb","Handler":null,"Name":"","Endpoint":""} Jul 27 23:37:02.895736 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.895715 2561 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 27 23:37:02.895868 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:02.895759 2561 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 27 23:37:03.188922 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:03.188836 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-bkkrs" event={"ID":"85cbe500-4ed4-4152-bc9c-0f325febf969","Type":"ContainerStarted","Data":"d4ca810ab66d7b56940bf0ebe99ea8faf864bdde7aaacc82f0184a1d35e8267f"} Jul 27 23:37:03.190896 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:03.190867 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" event={"ID":"c22c2085-c84d-4411-875e-4ea9a1d6686f","Type":"ContainerStarted","Data":"eebda8c337b1c36ba517ec7c35b6977893fbdc84dfe95fa89ea1382198421334"} Jul 27 23:37:03.202910 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:03.202875 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-bkkrs" podStartSLOduration=5.126556319 podStartE2EDuration="23.202861002s" totalImagesPullingTime="18.076304683s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:03.202635118 +0000 UTC m=+23.667017255" watchObservedRunningTime="2026-07-27 23:37:03.202861002 +0000 UTC m=+23.667243140" Jul 27 23:37:03.966952 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:03.966883 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:03.967121 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:03.967004 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:03.967272 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:03.967245 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:03.967406 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:03.967358 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:37:03.967406 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:03.967400 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:03.967519 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:03.967500 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:37:04.196738 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:04.196666 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" event={"ID":"c22c2085-c84d-4411-875e-4ea9a1d6686f","Type":"ContainerStarted","Data":"06e6a92382aebaec055905cb815713be0ca3e75d054ff391fbcf809e77d1d79b"} Jul 27 23:37:04.199858 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:04.199829 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"fa710e4cdaf975e48876b932ab4dcaa1135c298afd27dd91346466199a135768"} Jul 27 23:37:04.214237 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:04.214199 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ggpmg" podStartSLOduration=3.661258602 podStartE2EDuration="24.214185721s" totalImagesPullingTime="20.552927119s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:04.213526834 +0000 UTC m=+24.677908975" watchObservedRunningTime="2026-07-27 23:37:04.214185721 +0000 UTC m=+24.678567858" Jul 27 23:37:05.966880 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:05.966669 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:05.967471 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:05.966669 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:05.967471 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:05.966994 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:37:05.967471 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:05.966702 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:05.967471 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:05.967087 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:05.967471 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:05.967158 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:37:06.493679 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:06.493655 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:37:06.494190 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:06.494169 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:37:07.208880 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:07.208846 2561 generic.go:356] "Generic (PLEG): container finished" podID="50cc0ec6-df88-4006-bf61-2038cd76a81c" containerID="fd3138ab7b2ee589c8334a7a96a8fe57013ca2636f243e303e28131d5608d6ab" exitCode=0 Jul 27 23:37:07.209711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:07.208928 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerDied","Data":"fd3138ab7b2ee589c8334a7a96a8fe57013ca2636f243e303e28131d5608d6ab"} Jul 27 23:37:07.212746 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:07.212721 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" event={"ID":"5b97275d-8b9b-450a-9be4-5255067ffb36","Type":"ContainerStarted","Data":"952f05b39675fff6e86719ebaca454ec4184eade3fe61ce699f82ef75f5a9415"} Jul 27 23:37:07.252696 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:07.252656 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" podStartSLOduration=9.074174326 podStartE2EDuration="27.252643206s" totalImagesPullingTime="18.17846888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:07.252318764 +0000 UTC m=+27.716700902" watchObservedRunningTime="2026-07-27 23:37:07.252643206 +0000 UTC m=+27.717025348" Jul 27 23:37:07.966576 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:07.966545 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:07.966821 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:07.966551 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:07.966821 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:07.966645 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:37:07.966821 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:07.966738 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:37:07.966821 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:07.966800 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:07.967098 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:07.966879 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:08.082402 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.082374 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:37:08.083073 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.083035 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-xkg6r" Jul 27 23:37:08.215965 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.215934 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:37:08.216374 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.216029 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:37:08.216374 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.216094 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:37:08.233558 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.233530 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:37:08.233695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.233682 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:37:08.732072 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.732018 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4xtr"] Jul 27 23:37:08.732256 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.732195 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:08.732329 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:08.732297 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:08.735731 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.735706 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9b8f5"] Jul 27 23:37:08.735859 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.735811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:08.735913 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:08.735897 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:37:08.742660 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.742640 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-25t97"] Jul 27 23:37:08.742769 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:08.742715 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:08.742833 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:08.742779 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:37:09.219634 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:09.219469 2561 generic.go:356] "Generic (PLEG): container finished" podID="50cc0ec6-df88-4006-bf61-2038cd76a81c" containerID="8cc1c875ded1440201dcf109f9fe5162396b43560a03eb794fb33f49544a8229" exitCode=0 Jul 27 23:37:09.219988 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:09.219548 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerDied","Data":"8cc1c875ded1440201dcf109f9fe5162396b43560a03eb794fb33f49544a8229"} Jul 27 23:37:09.966827 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:09.966795 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:09.966979 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:09.966873 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:10.966311 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:10.966237 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:10.966875 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:10.966339 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:37:10.966875 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:10.966391 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:10.966875 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:10.966484 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:37:11.227212 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:11.227137 2561 generic.go:356] "Generic (PLEG): container finished" podID="50cc0ec6-df88-4006-bf61-2038cd76a81c" containerID="7e433dedef02ab9dfaef174e9094446709cbe0643e5282252ebcde1d524c886f" exitCode=0 Jul 27 23:37:11.227212 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:11.227180 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerDied","Data":"7e433dedef02ab9dfaef174e9094446709cbe0643e5282252ebcde1d524c886f"} Jul 27 23:37:11.966846 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:11.966810 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:11.967294 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:11.966944 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-d4xtr" podUID="fac2160a-9b05-4bfc-8e98-6205cbf4d252" Jul 27 23:37:12.965834 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:12.965799 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:12.966018 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:12.965912 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-9b8f5" podUID="a94ff868-7338-43d5-b8b0-49ebf8277545" Jul 27 23:37:12.966018 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:12.965974 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:12.966148 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:12.966099 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-25t97" podUID="d81e8510-04d0-4c8e-8b56-281f0ccdd165" Jul 27 23:37:13.592630 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.592596 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:13.593143 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.592679 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:13.593143 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.592776 2561 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:13.593143 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.592781 2561 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:13.593143 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.592888 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs podName:a94ff868-7338-43d5-b8b0-49ebf8277545 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.592836616 +0000 UTC m=+66.057218745 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs") pod "network-metrics-daemon-9b8f5" (UID: "a94ff868-7338-43d5-b8b0-49ebf8277545") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 27 23:37:13.593143 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.592928 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret podName:d81e8510-04d0-4c8e-8b56-281f0ccdd165 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.592910814 +0000 UTC m=+66.057292930 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret") pod "global-pull-secret-syncer-25t97" (UID: "d81e8510-04d0-4c8e-8b56-281f0ccdd165") : object "kube-system"/"original-pull-secret" not registered Jul 27 23:37:13.693803 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.693770 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:13.693991 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.693971 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 27 23:37:13.694088 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.694000 2561 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 27 23:37:13.694088 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.694013 2561 projected.go:196] Error preparing data for projected volume kube-api-access-b4sfb for pod openshift-network-diagnostics/network-check-target-d4xtr: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:13.694208 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:13.694094 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb podName:fac2160a-9b05-4bfc-8e98-6205cbf4d252 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.694074181 +0000 UTC m=+66.158456312 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-b4sfb" (UniqueName: "kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb") pod "network-check-target-d4xtr" (UID: "fac2160a-9b05-4bfc-8e98-6205cbf4d252") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 27 23:37:13.828762 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.828730 2561 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-203.ec2.internal" event="NodeReady" Jul 27 23:37:13.828918 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.828873 2561 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 27 23:37:13.860211 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.860146 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx"] Jul 27 23:37:13.865710 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.865683 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-55c85fbf77-4r5jv"] Jul 27 23:37:13.866369 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.866342 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" Jul 27 23:37:13.868511 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.868486 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-w5z4v\"" Jul 27 23:37:13.869149 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.869129 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-qcrwd"] Jul 27 23:37:13.869326 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.869305 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.870044 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.869546 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.870044 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.869812 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.872126 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.872105 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 27 23:37:13.872227 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.872129 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mkzjz\"" Jul 27 23:37:13.872478 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.872453 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4"] Jul 27 23:37:13.872795 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.872776 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:13.872931 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.872914 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 27 23:37:13.873587 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.873492 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 27 23:37:13.876489 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.876357 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7gmtp\"" Jul 27 23:37:13.876489 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.876453 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.876638 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.876502 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 27 23:37:13.876762 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.876740 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 27 23:37:13.877092 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.877043 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.878320 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.877483 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66"] Jul 27 23:37:13.878320 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.877953 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 27 23:37:13.878320 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.878005 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:13.881465 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.881440 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv"] Jul 27 23:37:13.887944 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.884183 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.887944 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.884437 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 27 23:37:13.887944 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.884865 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:13.887944 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.884888 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r4ssn\"" Jul 27 23:37:13.887944 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.887160 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.891490 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.891453 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 27 23:37:13.891887 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.891870 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 27 23:37:13.892280 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.892265 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5m48q\"" Jul 27 23:37:13.893983 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.893964 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.895833 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.895812 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 27 23:37:13.895984 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.895845 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.896340 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.896323 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-64znh"] Jul 27 23:37:13.896900 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.896446 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:13.898957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.898724 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 27 23:37:13.898957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.898817 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.898957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.898785 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 27 23:37:13.898957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.898847 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lxv9b\"" Jul 27 23:37:13.898957 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.898738 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.899819 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.899802 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn"] Jul 27 23:37:13.900293 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.900274 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:13.903867 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.903162 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.903867 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.903310 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 27 23:37:13.903867 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.903542 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 27 23:37:13.903867 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.903631 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mbstq\"" Jul 27 23:37:13.903867 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.903728 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.906340 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.906316 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx"] Jul 27 23:37:13.906473 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.906360 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6"] Jul 27 23:37:13.907163 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.907149 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:13.909519 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.909496 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r"] Jul 27 23:37:13.909647 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.909633 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:13.909731 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.909696 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 27 23:37:13.909872 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.909855 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 27 23:37:13.910006 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.909986 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 27 23:37:13.910258 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.910241 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.910445 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.910422 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.910534 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.910446 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9cf6n\"" Jul 27 23:37:13.912024 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.911978 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kmwr4\"" Jul 27 23:37:13.912264 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.912245 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jul 27 23:37:13.912345 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.912301 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jul 27 23:37:13.912637 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.912623 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv"] Jul 27 23:37:13.912711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.912653 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-qcrwd"] Jul 27 23:37:13.912711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.912674 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66"] Jul 27 23:37:13.912711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.912694 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6c94fdb577-j6wm7"] Jul 27 23:37:13.912854 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.912720 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" Jul 27 23:37:13.915278 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.915256 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.915464 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.915447 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bz27t\"" Jul 27 23:37:13.915570 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.915456 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.917411 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.917391 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-98c8q"] Jul 27 23:37:13.917619 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.917602 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:13.920347 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.919871 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.920551 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.920528 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zvkj8\"" Jul 27 23:37:13.920792 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.920771 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 27 23:37:13.920923 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.920903 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 27 23:37:13.921025 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.921007 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 27 23:37:13.921224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.921206 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 27 23:37:13.922823 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.922806 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p49sh"] Jul 27 23:37:13.922917 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.922899 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:13.923464 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.923395 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.925620 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.924993 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 27 23:37:13.925620 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.925372 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 27 23:37:13.925620 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.925445 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-fz2fq\"" Jul 27 23:37:13.925812 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.925685 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 27 23:37:13.927255 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927239 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4"] Jul 27 23:37:13.927331 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927268 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn"] Jul 27 23:37:13.927331 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927285 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55c85fbf77-4r5jv"] Jul 27 23:37:13.927441 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927352 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:13.927441 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927392 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r"] Jul 27 23:37:13.927441 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927428 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-64znh"] Jul 27 23:37:13.927578 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927449 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-98c8q"] Jul 27 23:37:13.927578 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927469 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c94fdb577-j6wm7"] Jul 27 23:37:13.927578 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927485 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6"] Jul 27 23:37:13.927578 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.927508 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p49sh"] Jul 27 23:37:13.929467 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.929420 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 27 23:37:13.929813 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.929794 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jk8w\"" Jul 27 23:37:13.929899 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.929835 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 27 23:37:13.965805 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.965784 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:13.967999 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.967977 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8p7f9\"" Jul 27 23:37:13.997451 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997427 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c410436e-064f-47ed-a826-87ae5903ca81-service-ca-bundle\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:13.997570 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997471 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1de2e520-a933-480d-beb5-98da93fc00c1-serving-cert\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:13.997570 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997492 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbs2p\" (UniqueName: \"kubernetes.io/projected/11cb99ed-8d79-48e3-9691-fc12bb0385d2-kube-api-access-fbs2p\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:13.997570 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997546 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-bound-sa-token\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.997723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997585 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11cb99ed-8d79-48e3-9691-fc12bb0385d2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:13.997723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997644 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-installation-pull-secrets\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.997723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997662 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c410436e-064f-47ed-a826-87ae5903ca81-tmp\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:13.997723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997677 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/feae3162-3781-4313-ac77-1d847639159b-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:13.997723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997699 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4xhz\" (UniqueName: \"kubernetes.io/projected/1de2e520-a933-480d-beb5-98da93fc00c1-kube-api-access-n4xhz\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:13.997920 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997731 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:13.997920 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997815 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptjb5\" (UniqueName: \"kubernetes.io/projected/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-kube-api-access-ptjb5\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:13.997920 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997850 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:13.997920 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997898 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ncvw\" (UniqueName: \"kubernetes.io/projected/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-kube-api-access-6ncvw\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:13.998081 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997949 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96g8z\" (UniqueName: \"kubernetes.io/projected/5d07c216-2c34-40b1-be08-bf1ae2201f45-kube-api-access-96g8z\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:13.998081 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.997975 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.998081 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998000 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:13.998081 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998023 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-certificates\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.998081 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998051 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c410436e-064f-47ed-a826-87ae5903ca81-snapshots\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998111 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbf7n\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-kube-api-access-gbf7n\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998134 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c410436e-064f-47ed-a826-87ae5903ca81-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998149 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1de2e520-a933-480d-beb5-98da93fc00c1-config\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998165 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7s69\" (UniqueName: \"kubernetes.io/projected/c410436e-064f-47ed-a826-87ae5903ca81-kube-api-access-m7s69\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998188 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1de2e520-a933-480d-beb5-98da93fc00c1-trusted-ca\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998210 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11cb99ed-8d79-48e3-9691-fc12bb0385d2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998234 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-config\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998256 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-image-registry-private-configuration\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.998281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998272 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:13.998601 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998291 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-trusted-ca\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.998601 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998346 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8gt2\" (UniqueName: \"kubernetes.io/projected/39fa12ca-dacb-413c-8d8c-6d8c2a00ba11-kube-api-access-k8gt2\") pod \"volume-data-source-validator-549c6d54cf-zckxx\" (UID: \"39fa12ca-dacb-413c-8d8c-6d8c2a00ba11\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" Jul 27 23:37:13.998601 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998374 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2714f10c-b53e-4923-be6f-941f9f8d060d-ca-trust-extracted\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:13.998601 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998394 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c410436e-064f-47ed-a826-87ae5903ca81-serving-cert\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:13.998601 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:13.998409 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:14.099486 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099455 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.099665 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099497 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:14.099665 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099523 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-default-certificate\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.099665 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.099617 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 23:37:14.099665 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.099639 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c85fbf77-4r5jv: secret "image-registry-tls" not found Jul 27 23:37:14.099665 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099643 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-tmp-dir\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.099688 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls podName:2714f10c-b53e-4923-be6f-941f9f8d060d nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.599672293 +0000 UTC m=+35.064054409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls") pod "image-registry-55c85fbf77-4r5jv" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d") : secret "image-registry-tls" not found Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099708 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-certificates\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099742 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c410436e-064f-47ed-a826-87ae5903ca81-snapshots\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099770 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbf7n\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-kube-api-access-gbf7n\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099792 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c410436e-064f-47ed-a826-87ae5903ca81-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099818 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1de2e520-a933-480d-beb5-98da93fc00c1-config\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-stats-auth\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099865 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7s69\" (UniqueName: \"kubernetes.io/projected/c410436e-064f-47ed-a826-87ae5903ca81-kube-api-access-m7s69\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099887 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1de2e520-a933-480d-beb5-98da93fc00c1-trusted-ca\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099910 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11cb99ed-8d79-48e3-9691-fc12bb0385d2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:14.099973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099953 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-config\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.099990 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7cmt\" (UniqueName: \"kubernetes.io/projected/2e9f1d79-4550-4983-8c6c-6677b0f82347-kube-api-access-m7cmt\") pod \"ingress-canary-98c8q\" (UID: \"2e9f1d79-4550-4983-8c6c-6677b0f82347\") " pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100025 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-image-registry-private-configuration\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100081 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100109 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-trusted-ca\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100139 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k84g\" (UniqueName: \"kubernetes.io/projected/960554c4-9d24-4732-bafa-3d829e451e2a-kube-api-access-6k84g\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100168 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8gt2\" (UniqueName: \"kubernetes.io/projected/39fa12ca-dacb-413c-8d8c-6d8c2a00ba11-kube-api-access-k8gt2\") pod \"volume-data-source-validator-549c6d54cf-zckxx\" (UID: \"39fa12ca-dacb-413c-8d8c-6d8c2a00ba11\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100199 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2714f10c-b53e-4923-be6f-941f9f8d060d-ca-trust-extracted\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100325 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-certificates\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100379 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/c410436e-064f-47ed-a826-87ae5903ca81-snapshots\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100475 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.100523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100511 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c410436e-064f-47ed-a826-87ae5903ca81-serving-cert\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100529 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/11cb99ed-8d79-48e3-9691-fc12bb0385d2-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100539 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2714f10c-b53e-4923-be6f-941f9f8d060d-ca-trust-extracted\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100545 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.100629 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.100679 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls podName:e16b04e5-d4a4-40e5-8479-79295c0c2ae1 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.600661051 +0000 UTC m=+35.065043167 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-vnp66" (UID: "e16b04e5-d4a4-40e5-8479-79295c0c2ae1") : secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100754 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c410436e-064f-47ed-a826-87ae5903ca81-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100832 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cj9sq\" (UniqueName: \"kubernetes.io/projected/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-kube-api-access-cj9sq\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100877 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c410436e-064f-47ed-a826-87ae5903ca81-service-ca-bundle\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1de2e520-a933-480d-beb5-98da93fc00c1-serving-cert\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100955 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbs2p\" (UniqueName: \"kubernetes.io/projected/11cb99ed-8d79-48e3-9691-fc12bb0385d2-kube-api-access-fbs2p\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.100978 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e9f1d79-4550-4983-8c6c-6677b0f82347-cert\") pod \"ingress-canary-98c8q\" (UID: \"2e9f1d79-4550-4983-8c6c-6677b0f82347\") " pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101003 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-bound-sa-token\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.101083 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101029 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11cb99ed-8d79-48e3-9691-fc12bb0385d2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101095 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-installation-pull-secrets\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101121 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c410436e-064f-47ed-a826-87ae5903ca81-tmp\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101149 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/feae3162-3781-4313-ac77-1d847639159b-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101176 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101200 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/1de2e520-a933-480d-beb5-98da93fc00c1-trusted-ca\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101210 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4xhz\" (UniqueName: \"kubernetes.io/projected/1de2e520-a933-480d-beb5-98da93fc00c1-kube-api-access-n4xhz\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101249 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101300 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptjb5\" (UniqueName: \"kubernetes.io/projected/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-kube-api-access-ptjb5\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101328 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101357 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4b9h\" (UniqueName: \"kubernetes.io/projected/adbed5e8-6042-4cc1-9217-9b9c79b12bd2-kube-api-access-t4b9h\") pod \"network-check-source-7d88cb55d8-6lq5r\" (UID: \"adbed5e8-6042-4cc1-9217-9b9c79b12bd2\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101377 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/1de2e520-a933-480d-beb5-98da93fc00c1-config\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101382 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-config-volume\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101441 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-trusted-ca\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101500 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c410436e-064f-47ed-a826-87ae5903ca81-service-ca-bundle\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.101525 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 23:37:14.101703 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.101582 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls podName:5d07c216-2c34-40b1-be08-bf1ae2201f45 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.601565491 +0000 UTC m=+35.065947616 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls") pod "cluster-samples-operator-6748489656-m84w4" (UID: "5d07c216-2c34-40b1-be08-bf1ae2201f45") : secret "samples-operator-tls" not found Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.101644 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.101678 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert podName:feae3162-3781-4313-ac77-1d847639159b nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.601667211 +0000 UTC m=+35.066049328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-6qxg6" (UID: "feae3162-3781-4313-ac77-1d847639159b") : secret "networking-console-plugin-cert" not found Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101858 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-config\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ncvw\" (UniqueName: \"kubernetes.io/projected/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-kube-api-access-6ncvw\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101933 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.101965 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96g8z\" (UniqueName: \"kubernetes.io/projected/5d07c216-2c34-40b1-be08-bf1ae2201f45-kube-api-access-96g8z\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.102031 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:14.102523 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.102331 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c410436e-064f-47ed-a826-87ae5903ca81-tmp\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.102872 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.102659 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/feae3162-3781-4313-ac77-1d847639159b-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:14.105257 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.105222 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/11cb99ed-8d79-48e3-9691-fc12bb0385d2-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:14.105257 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.105248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c410436e-064f-47ed-a826-87ae5903ca81-serving-cert\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.105424 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.105285 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-serving-cert\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:14.105530 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.105508 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-image-registry-private-configuration\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.105597 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.105544 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1de2e520-a933-480d-beb5-98da93fc00c1-serving-cert\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.107153 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.107133 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-installation-pull-secrets\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.109831 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.109806 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbf7n\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-kube-api-access-gbf7n\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.110287 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.110237 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8gt2\" (UniqueName: \"kubernetes.io/projected/39fa12ca-dacb-413c-8d8c-6d8c2a00ba11-kube-api-access-k8gt2\") pod \"volume-data-source-validator-549c6d54cf-zckxx\" (UID: \"39fa12ca-dacb-413c-8d8c-6d8c2a00ba11\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" Jul 27 23:37:14.111752 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.111620 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4xhz\" (UniqueName: \"kubernetes.io/projected/1de2e520-a933-480d-beb5-98da93fc00c1-kube-api-access-n4xhz\") pod \"console-operator-584dcb6b54-64znh\" (UID: \"1de2e520-a933-480d-beb5-98da93fc00c1\") " pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.111752 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.111690 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7s69\" (UniqueName: \"kubernetes.io/projected/c410436e-064f-47ed-a826-87ae5903ca81-kube-api-access-m7s69\") pod \"insights-operator-5885fdb759-qcrwd\" (UID: \"c410436e-064f-47ed-a826-87ae5903ca81\") " pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.112601 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.112567 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbs2p\" (UniqueName: \"kubernetes.io/projected/11cb99ed-8d79-48e3-9691-fc12bb0385d2-kube-api-access-fbs2p\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn\" (UID: \"11cb99ed-8d79-48e3-9691-fc12bb0385d2\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:14.112857 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.112837 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96g8z\" (UniqueName: \"kubernetes.io/projected/5d07c216-2c34-40b1-be08-bf1ae2201f45-kube-api-access-96g8z\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:14.115510 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.113502 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-bound-sa-token\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.115510 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.114015 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ncvw\" (UniqueName: \"kubernetes.io/projected/daab6831-caa2-4b41-9ee7-f8ffc5d6000b-kube-api-access-6ncvw\") pod \"service-ca-operator-5d6c75f84f-gg8vv\" (UID: \"daab6831-caa2-4b41-9ee7-f8ffc5d6000b\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:14.115951 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.115930 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptjb5\" (UniqueName: \"kubernetes.io/projected/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-kube-api-access-ptjb5\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:14.183138 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.183101 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" Jul 27 23:37:14.202515 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202484 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e9f1d79-4550-4983-8c6c-6677b0f82347-cert\") pod \"ingress-canary-98c8q\" (UID: \"2e9f1d79-4550-4983-8c6c-6677b0f82347\") " pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:14.202635 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202548 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.202635 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202595 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4b9h\" (UniqueName: \"kubernetes.io/projected/adbed5e8-6042-4cc1-9217-9b9c79b12bd2-kube-api-access-t4b9h\") pod \"network-check-source-7d88cb55d8-6lq5r\" (UID: \"adbed5e8-6042-4cc1-9217-9b9c79b12bd2\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" Jul 27 23:37:14.202635 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202613 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-config-volume\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.202635 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202632 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.202819 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202681 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-default-certificate\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.202819 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202707 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-tmp-dir\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.202819 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.202747 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.702723064 +0000 UTC m=+35.167105183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : configmap references non-existent config key: service-ca.crt Jul 27 23:37:14.202819 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.202776 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 23:37:14.202819 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202796 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-stats-auth\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.203043 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202830 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7cmt\" (UniqueName: \"kubernetes.io/projected/2e9f1d79-4550-4983-8c6c-6677b0f82347-kube-api-access-m7cmt\") pod \"ingress-canary-98c8q\" (UID: \"2e9f1d79-4550-4983-8c6c-6677b0f82347\") " pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:14.203043 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.202843 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.702825928 +0000 UTC m=+35.167208048 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : secret "router-metrics-certs-default" not found Jul 27 23:37:14.203043 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202897 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k84g\" (UniqueName: \"kubernetes.io/projected/960554c4-9d24-4732-bafa-3d829e451e2a-kube-api-access-6k84g\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.203043 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202933 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.203043 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.202978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cj9sq\" (UniqueName: \"kubernetes.io/projected/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-kube-api-access-cj9sq\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.203326 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.203233 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 23:37:14.203326 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.203241 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-config-volume\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.203421 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.203326 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls podName:1bbd5224-1723-4c96-a1d3-1a454db4cfa8 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:14.703310505 +0000 UTC m=+35.167692623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls") pod "dns-default-p49sh" (UID: "1bbd5224-1723-4c96-a1d3-1a454db4cfa8") : secret "dns-default-metrics-tls" not found Jul 27 23:37:14.203501 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.203484 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-tmp-dir\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.205315 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.205288 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2e9f1d79-4550-4983-8c6c-6677b0f82347-cert\") pod \"ingress-canary-98c8q\" (UID: \"2e9f1d79-4550-4983-8c6c-6677b0f82347\") " pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:14.205528 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.205512 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-stats-auth\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.205699 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.205681 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-default-certificate\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.211468 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.211402 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cj9sq\" (UniqueName: \"kubernetes.io/projected/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-kube-api-access-cj9sq\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.211548 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.211491 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k84g\" (UniqueName: \"kubernetes.io/projected/960554c4-9d24-4732-bafa-3d829e451e2a-kube-api-access-6k84g\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.211731 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.211697 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7cmt\" (UniqueName: \"kubernetes.io/projected/2e9f1d79-4550-4983-8c6c-6677b0f82347-kube-api-access-m7cmt\") pod \"ingress-canary-98c8q\" (UID: \"2e9f1d79-4550-4983-8c6c-6677b0f82347\") " pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:14.212315 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.212292 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4b9h\" (UniqueName: \"kubernetes.io/projected/adbed5e8-6042-4cc1-9217-9b9c79b12bd2-kube-api-access-t4b9h\") pod \"network-check-source-7d88cb55d8-6lq5r\" (UID: \"adbed5e8-6042-4cc1-9217-9b9c79b12bd2\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" Jul 27 23:37:14.226816 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.226799 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-qcrwd" Jul 27 23:37:14.235648 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.235630 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" Jul 27 23:37:14.241328 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.241309 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:14.250014 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.249996 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" Jul 27 23:37:14.262873 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.262857 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" Jul 27 23:37:14.275427 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.275407 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-98c8q" Jul 27 23:37:14.606807 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.606769 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.606844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.606883 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.606940 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.606963 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c85fbf77-4r5jv: secret "image-registry-tls" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.606972 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.607010 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.607029 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls podName:e16b04e5-d4a4-40e5-8479-79295c0c2ae1 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:15.607012653 +0000 UTC m=+36.071394768 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-vnp66" (UID: "e16b04e5-d4a4-40e5-8479-79295c0c2ae1") : secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.607079 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls podName:5d07c216-2c34-40b1-be08-bf1ae2201f45 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:15.607047501 +0000 UTC m=+36.071429617 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls") pod "cluster-samples-operator-6748489656-m84w4" (UID: "5d07c216-2c34-40b1-be08-bf1ae2201f45") : secret "samples-operator-tls" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.607118 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.607176 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls podName:2714f10c-b53e-4923-be6f-941f9f8d060d nodeName:}" failed. No retries permitted until 2026-07-27 23:37:15.607155849 +0000 UTC m=+36.071537984 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls") pod "image-registry-55c85fbf77-4r5jv" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d") : secret "image-registry-tls" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.607200 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 23:37:14.607377 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.607232 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert podName:feae3162-3781-4313-ac77-1d847639159b nodeName:}" failed. No retries permitted until 2026-07-27 23:37:15.607221076 +0000 UTC m=+36.071603192 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-6qxg6" (UID: "feae3162-3781-4313-ac77-1d847639159b") : secret "networking-console-plugin-cert" not found Jul 27 23:37:14.708459 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.708248 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.708459 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.708414 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:15.708391151 +0000 UTC m=+36.172773270 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : configmap references non-existent config key: service-ca.crt Jul 27 23:37:14.708672 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.708508 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:14.708672 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.708619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:14.708672 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.708660 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 23:37:14.708816 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.708717 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:15.70870504 +0000 UTC m=+36.173087155 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : secret "router-metrics-certs-default" not found Jul 27 23:37:14.708816 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.708752 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 23:37:14.708816 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:14.708806 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls podName:1bbd5224-1723-4c96-a1d3-1a454db4cfa8 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:15.708791504 +0000 UTC m=+36.173173715 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls") pod "dns-default-p49sh" (UID: "1bbd5224-1723-4c96-a1d3-1a454db4cfa8") : secret "dns-default-metrics-tls" not found Jul 27 23:37:14.966362 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.966286 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:14.966540 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.966287 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:14.968890 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.968866 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 23:37:14.969018 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.968977 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t92bh\"" Jul 27 23:37:14.969136 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:14.969116 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 23:37:15.597916 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.597845 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv"] Jul 27 23:37:15.600401 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.600326 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-qcrwd"] Jul 27 23:37:15.603857 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:15.603815 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaab6831_caa2_4b41_9ee7_f8ffc5d6000b.slice/crio-ded1eeb1d166758bcf6763f9458405b5c088bf97648bbd10c60b355a00cdd11e WatchSource:0}: Error finding container ded1eeb1d166758bcf6763f9458405b5c088bf97648bbd10c60b355a00cdd11e: Status 404 returned error can't find the container with id ded1eeb1d166758bcf6763f9458405b5c088bf97648bbd10c60b355a00cdd11e Jul 27 23:37:15.604917 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:15.604889 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc410436e_064f_47ed_a826_87ae5903ca81.slice/crio-cf15ac0658f14f6f2f4dfabcb1df3d04d8d51f22e63f230445f2bfddc47ac1f8 WatchSource:0}: Error finding container cf15ac0658f14f6f2f4dfabcb1df3d04d8d51f22e63f230445f2bfddc47ac1f8: Status 404 returned error can't find the container with id cf15ac0658f14f6f2f4dfabcb1df3d04d8d51f22e63f230445f2bfddc47ac1f8 Jul 27 23:37:15.612318 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.612290 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-64znh"] Jul 27 23:37:15.615209 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.615134 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r"] Jul 27 23:37:15.616440 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.616422 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-98c8q"] Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.616918 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.617045 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.617114 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.617169 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617184 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617268 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls podName:5d07c216-2c34-40b1-be08-bf1ae2201f45 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:17.617246735 +0000 UTC m=+38.081628854 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls") pod "cluster-samples-operator-6748489656-m84w4" (UID: "5d07c216-2c34-40b1-be08-bf1ae2201f45") : secret "samples-operator-tls" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617289 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617340 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls podName:e16b04e5-d4a4-40e5-8479-79295c0c2ae1 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:17.617324203 +0000 UTC m=+38.081706322 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-vnp66" (UID: "e16b04e5-d4a4-40e5-8479-79295c0c2ae1") : secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617371 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617409 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617413 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert podName:feae3162-3781-4313-ac77-1d847639159b nodeName:}" failed. No retries permitted until 2026-07-27 23:37:17.61740079 +0000 UTC m=+38.081782909 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-6qxg6" (UID: "feae3162-3781-4313-ac77-1d847639159b") : secret "networking-console-plugin-cert" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617420 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c85fbf77-4r5jv: secret "image-registry-tls" not found Jul 27 23:37:15.617579 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.617470 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls podName:2714f10c-b53e-4923-be6f-941f9f8d060d nodeName:}" failed. No retries permitted until 2026-07-27 23:37:17.617441561 +0000 UTC m=+38.081823684 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls") pod "image-registry-55c85fbf77-4r5jv" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d") : secret "image-registry-tls" not found Jul 27 23:37:15.618325 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.618099 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx"] Jul 27 23:37:15.620764 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:15.620716 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbed5e8_6042_4cc1_9217_9b9c79b12bd2.slice/crio-f2e7d87d90cdbf311fbf9fc77e89e81043214c1515de757b339bef3e773861db WatchSource:0}: Error finding container f2e7d87d90cdbf311fbf9fc77e89e81043214c1515de757b339bef3e773861db: Status 404 returned error can't find the container with id f2e7d87d90cdbf311fbf9fc77e89e81043214c1515de757b339bef3e773861db Jul 27 23:37:15.621571 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:15.621547 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39fa12ca_dacb_413c_8d8c_6d8c2a00ba11.slice/crio-bd1038239b76aa8581b19e59d57a47494d3eeeec5d804d5d6d8ab34cc15df390 WatchSource:0}: Error finding container bd1038239b76aa8581b19e59d57a47494d3eeeec5d804d5d6d8ab34cc15df390: Status 404 returned error can't find the container with id bd1038239b76aa8581b19e59d57a47494d3eeeec5d804d5d6d8ab34cc15df390 Jul 27 23:37:15.622605 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:15.622583 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e9f1d79_4550_4983_8c6c_6677b0f82347.slice/crio-9693ea999c959bcb55e1e6c2f7ea8e35fbfd3000dcd54800a49edad6babde98a WatchSource:0}: Error finding container 9693ea999c959bcb55e1e6c2f7ea8e35fbfd3000dcd54800a49edad6babde98a: Status 404 returned error can't find the container with id 9693ea999c959bcb55e1e6c2f7ea8e35fbfd3000dcd54800a49edad6babde98a Jul 27 23:37:15.626967 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.626250 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn"] Jul 27 23:37:15.631605 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:15.631585 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11cb99ed_8d79_48e3_9691_fc12bb0385d2.slice/crio-2404fca7ad4e062770003590a35c03f4b98c2fa29e8839c81ec7739c0f007483 WatchSource:0}: Error finding container 2404fca7ad4e062770003590a35c03f4b98c2fa29e8839c81ec7739c0f007483: Status 404 returned error can't find the container with id 2404fca7ad4e062770003590a35c03f4b98c2fa29e8839c81ec7739c0f007483 Jul 27 23:37:15.717687 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.717664 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:15.717766 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.717728 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:15.717803 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:15.717782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:15.717833 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.717815 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 23:37:15.717867 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.717854 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 23:37:15.717902 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.717883 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:17.717867301 +0000 UTC m=+38.182249418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : configmap references non-existent config key: service-ca.crt Jul 27 23:37:15.717945 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.717908 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:17.717899953 +0000 UTC m=+38.182282069 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : secret "router-metrics-certs-default" not found Jul 27 23:37:15.717945 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:15.717918 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls podName:1bbd5224-1723-4c96-a1d3-1a454db4cfa8 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:17.717912862 +0000 UTC m=+38.182294977 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls") pod "dns-default-p49sh" (UID: "1bbd5224-1723-4c96-a1d3-1a454db4cfa8") : secret "dns-default-metrics-tls" not found Jul 27 23:37:16.251649 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.251611 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-98c8q" event={"ID":"2e9f1d79-4550-4983-8c6c-6677b0f82347","Type":"ContainerStarted","Data":"9693ea999c959bcb55e1e6c2f7ea8e35fbfd3000dcd54800a49edad6babde98a"} Jul 27 23:37:16.254090 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.254010 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" event={"ID":"1de2e520-a933-480d-beb5-98da93fc00c1","Type":"ContainerStarted","Data":"649fe50af71adfdd425643ffdd7ab22492f11a202ba0e1304f49a635836ca723"} Jul 27 23:37:16.256039 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.255966 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" event={"ID":"daab6831-caa2-4b41-9ee7-f8ffc5d6000b","Type":"ContainerStarted","Data":"ded1eeb1d166758bcf6763f9458405b5c088bf97648bbd10c60b355a00cdd11e"} Jul 27 23:37:16.257908 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.257873 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" event={"ID":"adbed5e8-6042-4cc1-9217-9b9c79b12bd2","Type":"ContainerStarted","Data":"f2e7d87d90cdbf311fbf9fc77e89e81043214c1515de757b339bef3e773861db"} Jul 27 23:37:16.259276 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.259251 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qcrwd" event={"ID":"c410436e-064f-47ed-a826-87ae5903ca81","Type":"ContainerStarted","Data":"cf15ac0658f14f6f2f4dfabcb1df3d04d8d51f22e63f230445f2bfddc47ac1f8"} Jul 27 23:37:16.263236 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.263206 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" event={"ID":"11cb99ed-8d79-48e3-9691-fc12bb0385d2","Type":"ContainerStarted","Data":"2404fca7ad4e062770003590a35c03f4b98c2fa29e8839c81ec7739c0f007483"} Jul 27 23:37:16.267411 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.267332 2561 generic.go:356] "Generic (PLEG): container finished" podID="50cc0ec6-df88-4006-bf61-2038cd76a81c" containerID="50ad956e0651a7b9d3030bae93b17ca0a55ff28914647fedca6ec672f6ca0693" exitCode=0 Jul 27 23:37:16.267495 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.267411 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerDied","Data":"50ad956e0651a7b9d3030bae93b17ca0a55ff28914647fedca6ec672f6ca0693"} Jul 27 23:37:16.270280 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:16.270205 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" event={"ID":"39fa12ca-dacb-413c-8d8c-6d8c2a00ba11","Type":"ContainerStarted","Data":"bd1038239b76aa8581b19e59d57a47494d3eeeec5d804d5d6d8ab34cc15df390"} Jul 27 23:37:17.285620 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.285371 2561 generic.go:356] "Generic (PLEG): container finished" podID="50cc0ec6-df88-4006-bf61-2038cd76a81c" containerID="9e4a57ce20ae3873ad91113a6e42633e39ee9aeedee87811f07e775bd858924b" exitCode=0 Jul 27 23:37:17.286123 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.285673 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerDied","Data":"9e4a57ce20ae3873ad91113a6e42633e39ee9aeedee87811f07e775bd858924b"} Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.637689 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.637820 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.637868 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.637923 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638081 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638148 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls podName:e16b04e5-d4a4-40e5-8479-79295c0c2ae1 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:21.638128849 +0000 UTC m=+42.102510970 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-vnp66" (UID: "e16b04e5-d4a4-40e5-8479-79295c0c2ae1") : secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638539 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638584 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls podName:5d07c216-2c34-40b1-be08-bf1ae2201f45 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:21.638570034 +0000 UTC m=+42.102952153 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls") pod "cluster-samples-operator-6748489656-m84w4" (UID: "5d07c216-2c34-40b1-be08-bf1ae2201f45") : secret "samples-operator-tls" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638653 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638685 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert podName:feae3162-3781-4313-ac77-1d847639159b nodeName:}" failed. No retries permitted until 2026-07-27 23:37:21.638673133 +0000 UTC m=+42.103055255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-6qxg6" (UID: "feae3162-3781-4313-ac77-1d847639159b") : secret "networking-console-plugin-cert" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638739 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638749 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c85fbf77-4r5jv: secret "image-registry-tls" not found Jul 27 23:37:17.638869 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.638778 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls podName:2714f10c-b53e-4923-be6f-941f9f8d060d nodeName:}" failed. No retries permitted until 2026-07-27 23:37:21.638768992 +0000 UTC m=+42.103151109 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls") pod "image-registry-55c85fbf77-4r5jv" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d") : secret "image-registry-tls" not found Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.738386 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.738493 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:17.738550 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.738687 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.738746 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:21.73872806 +0000 UTC m=+42.203110193 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : secret "router-metrics-certs-default" not found Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.739166 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.739216 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls podName:1bbd5224-1723-4c96-a1d3-1a454db4cfa8 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:21.739200624 +0000 UTC m=+42.203582744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls") pod "dns-default-p49sh" (UID: "1bbd5224-1723-4c96-a1d3-1a454db4cfa8") : secret "dns-default-metrics-tls" not found Jul 27 23:37:17.739326 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:17.739285 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:21.739274789 +0000 UTC m=+42.203656910 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : configmap references non-existent config key: service-ca.crt Jul 27 23:37:21.674207 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:21.674136 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:21.674207 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:21.674209 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:21.674259 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674294 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:21.674312 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674357 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674378 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c85fbf77-4r5jv: secret "image-registry-tls" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674413 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674420 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674363 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert podName:feae3162-3781-4313-ac77-1d847639159b nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.674347054 +0000 UTC m=+50.138729170 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-6qxg6" (UID: "feae3162-3781-4313-ac77-1d847639159b") : secret "networking-console-plugin-cert" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674479 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls podName:2714f10c-b53e-4923-be6f-941f9f8d060d nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.674462687 +0000 UTC m=+50.138844808 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls") pod "image-registry-55c85fbf77-4r5jv" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d") : secret "image-registry-tls" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674498 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls podName:5d07c216-2c34-40b1-be08-bf1ae2201f45 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.674490531 +0000 UTC m=+50.138872647 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls") pod "cluster-samples-operator-6748489656-m84w4" (UID: "5d07c216-2c34-40b1-be08-bf1ae2201f45") : secret "samples-operator-tls" not found Jul 27 23:37:21.674940 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.674519 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls podName:e16b04e5-d4a4-40e5-8479-79295c0c2ae1 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.674509979 +0000 UTC m=+50.138892094 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-vnp66" (UID: "e16b04e5-d4a4-40e5-8479-79295c0c2ae1") : secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:21.775707 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:21.775674 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:21.775890 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:21.775749 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:21.775890 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:21.775786 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:21.775890 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.775843 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 23:37:21.775890 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.775878 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 23:37:21.776051 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.775908 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls podName:1bbd5224-1723-4c96-a1d3-1a454db4cfa8 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.775889649 +0000 UTC m=+50.240271766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls") pod "dns-default-p49sh" (UID: "1bbd5224-1723-4c96-a1d3-1a454db4cfa8") : secret "dns-default-metrics-tls" not found Jul 27 23:37:21.776051 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.775929 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.775917653 +0000 UTC m=+50.240299775 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : secret "router-metrics-certs-default" not found Jul 27 23:37:21.776051 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:21.775951 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.775938253 +0000 UTC m=+50.240320390 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : configmap references non-existent config key: service-ca.crt Jul 27 23:37:24.308049 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.308012 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" event={"ID":"daab6831-caa2-4b41-9ee7-f8ffc5d6000b","Type":"ContainerStarted","Data":"0b28514f30a7ea47cd92bbe87dbd35f596dde01956b51c96a3f8e4f83a2dd8f8"} Jul 27 23:37:24.309818 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.309790 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" event={"ID":"adbed5e8-6042-4cc1-9217-9b9c79b12bd2","Type":"ContainerStarted","Data":"b44bb76584cf47a28015ee00a94e42376caa84540754a8622d0748f9d885ec59"} Jul 27 23:37:24.311505 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.311475 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qcrwd" event={"ID":"c410436e-064f-47ed-a826-87ae5903ca81","Type":"ContainerStarted","Data":"628a57e4aa0a7cab400c5fd00f91efce553b7f1a6abfe8cc37881ee487f2d907"} Jul 27 23:37:24.313237 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.313212 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" event={"ID":"11cb99ed-8d79-48e3-9691-fc12bb0385d2","Type":"ContainerStarted","Data":"7873a81070c381025e31ef6964cd39b634cb40dce082526b8e81b5e939e13b64"} Jul 27 23:37:24.317086 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.317041 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" event={"ID":"50cc0ec6-df88-4006-bf61-2038cd76a81c","Type":"ContainerStarted","Data":"9f470fccd69b0cf44fb337246497f33cafe48976d8ee76fd9531e08b801d1b7c"} Jul 27 23:37:24.319036 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.319014 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" event={"ID":"39fa12ca-dacb-413c-8d8c-6d8c2a00ba11","Type":"ContainerStarted","Data":"39d61f1031b2ab78abf3895049e4a388e00f547612508db5a4fafa0a861d8481"} Jul 27 23:37:24.321377 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.321351 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-98c8q" event={"ID":"2e9f1d79-4550-4983-8c6c-6677b0f82347","Type":"ContainerStarted","Data":"13ac15fc346f47549a2fc563729d5ba50f14d1b2ceda3207d5ff94d5f3f7a7ea"} Jul 27 23:37:24.323847 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.323806 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" podStartSLOduration=29.574184516 podStartE2EDuration="37.323790664s" totalImagesPullingTime="7.749606148s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:24.323439769 +0000 UTC m=+44.787821910" watchObservedRunningTime="2026-07-27 23:37:24.323790664 +0000 UTC m=+44.788172802" Jul 27 23:37:24.323955 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.323934 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/0.log" Jul 27 23:37:24.324013 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.323984 2561 generic.go:356] "Generic (PLEG): container finished" podID="1de2e520-a933-480d-beb5-98da93fc00c1" containerID="394366455e5624cc379955dc00082ca4c2fa1affe33d7aea2cb70e5a18ad6d52" exitCode=255 Jul 27 23:37:24.324084 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.324013 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" event={"ID":"1de2e520-a933-480d-beb5-98da93fc00c1","Type":"ContainerDied","Data":"394366455e5624cc379955dc00082ca4c2fa1affe33d7aea2cb70e5a18ad6d52"} Jul 27 23:37:24.324207 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.324193 2561 scope.go:117] "RemoveContainer" containerID="394366455e5624cc379955dc00082ca4c2fa1affe33d7aea2cb70e5a18ad6d52" Jul 27 23:37:24.342292 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.342243 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" podStartSLOduration=29.620774222 podStartE2EDuration="37.342231114s" totalImagesPullingTime="7.721456892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:24.341486459 +0000 UTC m=+44.805868598" watchObservedRunningTime="2026-07-27 23:37:24.342231114 +0000 UTC m=+44.806613254" Jul 27 23:37:24.388598 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.388495 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-zckxx" podStartSLOduration=29.701005609 podStartE2EDuration="37.388478839s" totalImagesPullingTime="7.68747323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:24.388002271 +0000 UTC m=+44.852384410" watchObservedRunningTime="2026-07-27 23:37:24.388478839 +0000 UTC m=+44.852860976" Jul 27 23:37:24.388803 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.388773 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-qcrwd" podStartSLOduration=29.636094922 podStartE2EDuration="37.388759891s" totalImagesPullingTime="7.752664969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:24.358777072 +0000 UTC m=+44.823159209" watchObservedRunningTime="2026-07-27 23:37:24.388759891 +0000 UTC m=+44.853142010" Jul 27 23:37:24.404226 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.404151 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-6lq5r" podStartSLOduration=29.4000273 podStartE2EDuration="37.404134187s" totalImagesPullingTime="8.004106887s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:24.403676551 +0000 UTC m=+44.868058690" watchObservedRunningTime="2026-07-27 23:37:24.404134187 +0000 UTC m=+44.868516324" Jul 27 23:37:24.417815 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.417769 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-98c8q" podStartSLOduration=3.687897533 podStartE2EDuration="11.41775343s" totalImagesPullingTime="7.729855897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:24.417161886 +0000 UTC m=+44.881544026" watchObservedRunningTime="2026-07-27 23:37:24.41775343 +0000 UTC m=+44.882135573" Jul 27 23:37:24.442042 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:24.441415 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bqmzs" podStartSLOduration=15.775552812 podStartE2EDuration="44.441398121s" totalImagesPullingTime="28.665845309s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:24.439551868 +0000 UTC m=+44.903934007" watchObservedRunningTime="2026-07-27 23:37:24.441398121 +0000 UTC m=+44.905780260" Jul 27 23:37:25.331164 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:25.331131 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:37:25.331665 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:25.331652 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/0.log" Jul 27 23:37:25.331714 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:25.331689 2561 generic.go:356] "Generic (PLEG): container finished" podID="1de2e520-a933-480d-beb5-98da93fc00c1" containerID="0e1fb3c4135275c8a4f549f2f6a5b8035ccf1f48f7373c86cb05fc9253ae0b39" exitCode=255 Jul 27 23:37:25.331822 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:25.331795 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" event={"ID":"1de2e520-a933-480d-beb5-98da93fc00c1","Type":"ContainerDied","Data":"0e1fb3c4135275c8a4f549f2f6a5b8035ccf1f48f7373c86cb05fc9253ae0b39"} Jul 27 23:37:25.331905 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:25.331891 2561 scope.go:117] "RemoveContainer" containerID="394366455e5624cc379955dc00082ca4c2fa1affe33d7aea2cb70e5a18ad6d52" Jul 27 23:37:25.331995 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:25.331973 2561 scope.go:117] "RemoveContainer" containerID="0e1fb3c4135275c8a4f549f2f6a5b8035ccf1f48f7373c86cb05fc9253ae0b39" Jul 27 23:37:25.332539 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:25.332183 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-64znh_openshift-console-operator(1de2e520-a933-480d-beb5-98da93fc00c1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" podUID="1de2e520-a933-480d-beb5-98da93fc00c1" Jul 27 23:37:26.210822 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.210791 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-8n4wb"] Jul 27 23:37:26.243921 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.243896 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.244863 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.244841 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8n4wb"] Jul 27 23:37:26.246519 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.246492 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sbzb5\"" Jul 27 23:37:26.246643 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.246624 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 27 23:37:26.247337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.247317 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 27 23:37:26.318848 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.318816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2qbc\" (UniqueName: \"kubernetes.io/projected/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-api-access-g2qbc\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.318998 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.318896 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.318998 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.318972 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-crio-socket\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.319132 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.319006 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-data-volume\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.319132 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.319022 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.337544 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.337519 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:37:26.337879 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.337843 2561 scope.go:117] "RemoveContainer" containerID="0e1fb3c4135275c8a4f549f2f6a5b8035ccf1f48f7373c86cb05fc9253ae0b39" Jul 27 23:37:26.337995 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.337976 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-64znh_openshift-console-operator(1de2e520-a933-480d-beb5-98da93fc00c1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" podUID="1de2e520-a933-480d-beb5-98da93fc00c1" Jul 27 23:37:26.420284 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.420257 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2qbc\" (UniqueName: \"kubernetes.io/projected/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-api-access-g2qbc\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.420428 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.420301 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.420428 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.420360 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-crio-socket\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.420428 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.420401 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-data-volume\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.420428 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.420427 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.420636 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.420525 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 23:37:26.420636 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.420540 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 23:37:26.420636 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.420605 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:26.920587763 +0000 UTC m=+47.384969882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : configmap "kube-rbac-proxy" not found Jul 27 23:37:26.420636 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.420627 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:26.920620937 +0000 UTC m=+47.385003053 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : secret "insights-runtime-extractor-tls" not found Jul 27 23:37:26.420849 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.420686 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-crio-socket\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.420897 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.420852 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-data-volume\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.431969 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.431946 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2qbc\" (UniqueName: \"kubernetes.io/projected/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-api-access-g2qbc\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.924472 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.924431 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.924651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:26.924502 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:26.924651 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.924586 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 23:37:26.924651 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.924607 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 23:37:26.924651 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.924651 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:27.924636159 +0000 UTC m=+48.389018275 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : secret "insights-runtime-extractor-tls" not found Jul 27 23:37:26.924806 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:26.924665 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:27.924659294 +0000 UTC m=+48.389041410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : configmap "kube-rbac-proxy" not found Jul 27 23:37:27.932922 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:27.932885 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:27.933314 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:27.933001 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:27.933314 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:27.933009 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 23:37:27.933314 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:27.933103 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.933086732 +0000 UTC m=+50.397468848 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : configmap "kube-rbac-proxy" not found Jul 27 23:37:27.933314 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:27.933105 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 23:37:27.933314 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:27.933146 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:29.933135608 +0000 UTC m=+50.397517724 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : secret "insights-runtime-extractor-tls" not found Jul 27 23:37:28.794386 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:28.794357 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xmnf8_3eefe218-95cd-4c4d-b878-fc8864079d8b/dns-node-resolver/0.log" Jul 27 23:37:29.398166 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.398142 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5zbj9_b627ead5-83e7-472c-b2ac-1b126e15fc3c/node-ca/0.log" Jul 27 23:37:29.748226 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.748139 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:29.748226 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.748198 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:29.748425 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.748232 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:29.748425 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.748268 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:29.748425 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748302 2561 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jul 27 23:37:29.748425 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748364 2561 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:29.748425 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748402 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert podName:feae3162-3781-4313-ac77-1d847639159b nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.748362844 +0000 UTC m=+66.212744960 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-6qxg6" (UID: "feae3162-3781-4313-ac77-1d847639159b") : secret "networking-console-plugin-cert" not found Jul 27 23:37:29.748425 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748425 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls podName:e16b04e5-d4a4-40e5-8479-79295c0c2ae1 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.748412534 +0000 UTC m=+66.212794650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-vnp66" (UID: "e16b04e5-d4a4-40e5-8479-79295c0c2ae1") : secret "cluster-monitoring-operator-tls" not found Jul 27 23:37:29.748425 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748369 2561 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 27 23:37:29.748678 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748454 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls podName:5d07c216-2c34-40b1-be08-bf1ae2201f45 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.74844739 +0000 UTC m=+66.212829506 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls") pod "cluster-samples-operator-6748489656-m84w4" (UID: "5d07c216-2c34-40b1-be08-bf1ae2201f45") : secret "samples-operator-tls" not found Jul 27 23:37:29.748678 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748457 2561 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 27 23:37:29.748678 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748472 2561 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-55c85fbf77-4r5jv: secret "image-registry-tls" not found Jul 27 23:37:29.748678 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.748529 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls podName:2714f10c-b53e-4923-be6f-941f9f8d060d nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.748515095 +0000 UTC m=+66.212897228 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls") pod "image-registry-55c85fbf77-4r5jv" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d") : secret "image-registry-tls" not found Jul 27 23:37:29.848839 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.848805 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:29.848983 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.848867 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:29.848983 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.848970 2561 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 27 23:37:29.849093 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.848971 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:29.849093 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.848984 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.84896539 +0000 UTC m=+66.313347524 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : configmap references non-existent config key: service-ca.crt Jul 27 23:37:29.849093 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.849042 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs podName:960554c4-9d24-4732-bafa-3d829e451e2a nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.849026278 +0000 UTC m=+66.313408394 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs") pod "router-default-6c94fdb577-j6wm7" (UID: "960554c4-9d24-4732-bafa-3d829e451e2a") : secret "router-metrics-certs-default" not found Jul 27 23:37:29.849093 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.849075 2561 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 27 23:37:29.849230 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.849126 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls podName:1bbd5224-1723-4c96-a1d3-1a454db4cfa8 nodeName:}" failed. No retries permitted until 2026-07-27 23:37:45.849114417 +0000 UTC m=+66.313496552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls") pod "dns-default-p49sh" (UID: "1bbd5224-1723-4c96-a1d3-1a454db4cfa8") : secret "dns-default-metrics-tls" not found Jul 27 23:37:29.949454 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.949422 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:29.949613 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.949531 2561 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 27 23:37:29.949613 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:29.949575 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:29.949613 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.949592 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:33.949573919 +0000 UTC m=+54.413956039 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : configmap "kube-rbac-proxy" not found Jul 27 23:37:29.949789 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.949687 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 23:37:29.949789 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:29.949753 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:33.949735102 +0000 UTC m=+54.414117234 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : secret "insights-runtime-extractor-tls" not found Jul 27 23:37:30.195127 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:30.195101 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-98c8q_2e9f1d79-4550-4983-8c6c-6677b0f82347/serve-healthcheck-canary/0.log" Jul 27 23:37:30.996160 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:30.996129 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn_11cb99ed-8d79-48e3-9691-fc12bb0385d2/kube-storage-version-migrator-operator/0.log" Jul 27 23:37:33.984208 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:33.984169 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:33.984706 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:33.984260 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:33.984706 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:33.984325 2561 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 27 23:37:33.984706 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:33.984386 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls podName:34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c nodeName:}" failed. No retries permitted until 2026-07-27 23:37:41.984369387 +0000 UTC m=+62.448751505 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls") pod "insights-runtime-extractor-8n4wb" (UID: "34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c") : secret "insights-runtime-extractor-tls" not found Jul 27 23:37:33.984818 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:33.984743 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:34.241882 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:34.241800 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:34.241882 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:34.241851 2561 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:34.242192 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:34.242180 2561 scope.go:117] "RemoveContainer" containerID="0e1fb3c4135275c8a4f549f2f6a5b8035ccf1f48f7373c86cb05fc9253ae0b39" Jul 27 23:37:34.242342 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:37:34.242324 2561 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-64znh_openshift-console-operator(1de2e520-a933-480d-beb5-98da93fc00c1)\"" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" podUID="1de2e520-a933-480d-beb5-98da93fc00c1" Jul 27 23:37:40.242914 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:40.242886 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7rpxv" Jul 27 23:37:42.050667 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:42.050633 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:42.052818 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:42.052800 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-8n4wb\" (UID: \"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c\") " pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:42.156248 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:42.156219 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sbzb5\"" Jul 27 23:37:42.165218 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:42.165181 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-8n4wb" Jul 27 23:37:42.285069 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:42.285020 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-8n4wb"] Jul 27 23:37:42.288431 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:42.288403 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34f8e6fa_9d30_45fa_86ca_7e24a89d3a6c.slice/crio-ee3aac56f20a71a421a1293576bd1d48300ea515b43e548331e0c60f8420832d WatchSource:0}: Error finding container ee3aac56f20a71a421a1293576bd1d48300ea515b43e548331e0c60f8420832d: Status 404 returned error can't find the container with id ee3aac56f20a71a421a1293576bd1d48300ea515b43e548331e0c60f8420832d Jul 27 23:37:42.385789 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:42.385725 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n4wb" event={"ID":"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c","Type":"ContainerStarted","Data":"ee3aac56f20a71a421a1293576bd1d48300ea515b43e548331e0c60f8420832d"} Jul 27 23:37:44.967002 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:44.966967 2561 scope.go:117] "RemoveContainer" containerID="0e1fb3c4135275c8a4f549f2f6a5b8035ccf1f48f7373c86cb05fc9253ae0b39" Jul 27 23:37:45.682752 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.682714 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:45.682752 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.682760 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:45.685168 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.685140 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 27 23:37:45.685168 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.685159 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 27 23:37:45.695465 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.695439 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a94ff868-7338-43d5-b8b0-49ebf8277545-metrics-certs\") pod \"network-metrics-daemon-9b8f5\" (UID: \"a94ff868-7338-43d5-b8b0-49ebf8277545\") " pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:45.695465 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.695450 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/d81e8510-04d0-4c8e-8b56-281f0ccdd165-original-pull-secret\") pod \"global-pull-secret-syncer-25t97\" (UID: \"d81e8510-04d0-4c8e-8b56-281f0ccdd165\") " pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:45.783717 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.783677 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:45.783900 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.783725 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:45.783900 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.783770 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:45.783900 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.783817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:45.783900 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.783867 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:45.786907 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.786853 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/feae3162-3781-4313-ac77-1d847639159b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-6qxg6\" (UID: \"feae3162-3781-4313-ac77-1d847639159b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:45.787047 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.787015 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4sfb\" (UniqueName: \"kubernetes.io/projected/fac2160a-9b05-4bfc-8e98-6205cbf4d252-kube-api-access-b4sfb\") pod \"network-check-target-d4xtr\" (UID: \"fac2160a-9b05-4bfc-8e98-6205cbf4d252\") " pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:45.787169 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.787140 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/e16b04e5-d4a4-40e5-8479-79295c0c2ae1-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-vnp66\" (UID: \"e16b04e5-d4a4-40e5-8479-79295c0c2ae1\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:45.787372 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.787348 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"image-registry-55c85fbf77-4r5jv\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:45.787600 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.787576 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/5d07c216-2c34-40b1-be08-bf1ae2201f45-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-m84w4\" (UID: \"5d07c216-2c34-40b1-be08-bf1ae2201f45\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:45.790102 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.790081 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8p7f9\"" Jul 27 23:37:45.797624 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.797603 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:45.879142 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.879107 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-25t97" Jul 27 23:37:45.885115 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.885090 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:45.885247 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.885140 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:45.885247 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.885167 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:45.885792 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.885766 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/960554c4-9d24-4732-bafa-3d829e451e2a-service-ca-bundle\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:45.888319 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.888105 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t92bh\"" Jul 27 23:37:45.888319 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.888268 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/960554c4-9d24-4732-bafa-3d829e451e2a-metrics-certs\") pod \"router-default-6c94fdb577-j6wm7\" (UID: \"960554c4-9d24-4732-bafa-3d829e451e2a\") " pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:45.889210 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.889170 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bbd5224-1723-4c96-a1d3-1a454db4cfa8-metrics-tls\") pod \"dns-default-p49sh\" (UID: \"1bbd5224-1723-4c96-a1d3-1a454db4cfa8\") " pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:45.895704 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:45.895654 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-9b8f5" Jul 27 23:37:46.000152 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.000082 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-mkzjz\"" Jul 27 23:37:46.008250 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.008217 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:46.011497 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.011474 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-5m48q\"" Jul 27 23:37:46.020455 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.020432 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" Jul 27 23:37:46.031326 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.031214 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-r4ssn\"" Jul 27 23:37:46.039876 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.039851 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" Jul 27 23:37:46.058296 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.058276 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-kmwr4\"" Jul 27 23:37:46.066439 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.066421 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" Jul 27 23:37:46.072478 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.072451 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-zvkj8\"" Jul 27 23:37:46.080263 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.080245 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:46.084489 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.084469 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-2jk8w\"" Jul 27 23:37:46.093964 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:46.093887 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:47.406854 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.406288 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:37:47.406854 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.406358 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" event={"ID":"1de2e520-a933-480d-beb5-98da93fc00c1","Type":"ContainerStarted","Data":"632fead300baf901b0c9d84907e4043bf20fe9d03453ba83376b95021f8c1ae7"} Jul 27 23:37:47.409403 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.407712 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:47.410103 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.409767 2561 patch_prober.go:28] interesting pod/console-operator-584dcb6b54-64znh container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" start-of-body= Jul 27 23:37:47.410103 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.409819 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" podUID="1de2e520-a933-480d-beb5-98da93fc00c1" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.14:8443/readyz\": dial tcp 10.132.0.14:8443: connect: connection refused" Jul 27 23:37:47.423927 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.423873 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" podStartSLOduration=52.68835763 podStartE2EDuration="1m0.42385367s" totalImagesPullingTime="7.73549604s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:47.423624226 +0000 UTC m=+67.888006365" watchObservedRunningTime="2026-07-27 23:37:47.42385367 +0000 UTC m=+67.888235812" Jul 27 23:37:47.642892 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.642868 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66"] Jul 27 23:37:47.648924 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.648886 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p49sh"] Jul 27 23:37:47.650194 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.650082 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4"] Jul 27 23:37:47.650640 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:47.650613 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode16b04e5_d4a4_40e5_8479_79295c0c2ae1.slice/crio-b0824afa46ba5c7c0ef29184a68dfd0980c1ed1e5c26898af21eed1e3d76ab78 WatchSource:0}: Error finding container b0824afa46ba5c7c0ef29184a68dfd0980c1ed1e5c26898af21eed1e3d76ab78: Status 404 returned error can't find the container with id b0824afa46ba5c7c0ef29184a68dfd0980c1ed1e5c26898af21eed1e3d76ab78 Jul 27 23:37:47.654836 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:47.653788 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bbd5224_1723_4c96_a1d3_1a454db4cfa8.slice/crio-39343018ce6cea4519ade35cb1693117b4af0275b69eee9c2451e7d659c7d0a5 WatchSource:0}: Error finding container 39343018ce6cea4519ade35cb1693117b4af0275b69eee9c2451e7d659c7d0a5: Status 404 returned error can't find the container with id 39343018ce6cea4519ade35cb1693117b4af0275b69eee9c2451e7d659c7d0a5 Jul 27 23:37:47.655184 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.655163 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-25t97"] Jul 27 23:37:47.659610 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:47.659584 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd81e8510_04d0_4c8e_8b56_281f0ccdd165.slice/crio-21e4d5d225f165e4e92d5ab8c3f485dd9b913fe36b3d49109d8e7cf34dcf7dfd WatchSource:0}: Error finding container 21e4d5d225f165e4e92d5ab8c3f485dd9b913fe36b3d49109d8e7cf34dcf7dfd: Status 404 returned error can't find the container with id 21e4d5d225f165e4e92d5ab8c3f485dd9b913fe36b3d49109d8e7cf34dcf7dfd Jul 27 23:37:47.662879 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.662861 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-55c85fbf77-4r5jv"] Jul 27 23:37:47.665599 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:47.665580 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2714f10c_b53e_4923_be6f_941f9f8d060d.slice/crio-ed5fc150d518beb4e0f4656e90125580c4e715ef04a4930da59f2983f00cd669 WatchSource:0}: Error finding container ed5fc150d518beb4e0f4656e90125580c4e715ef04a4930da59f2983f00cd669: Status 404 returned error can't find the container with id ed5fc150d518beb4e0f4656e90125580c4e715ef04a4930da59f2983f00cd669 Jul 27 23:37:47.698730 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.698678 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-d4xtr"] Jul 27 23:37:47.701244 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.701222 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6c94fdb577-j6wm7"] Jul 27 23:37:47.746754 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.746680 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-9b8f5"] Jul 27 23:37:47.748826 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:47.748762 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda94ff868_7338_43d5_b8b0_49ebf8277545.slice/crio-47cbac0ca7e5fdd0511ae4c06edf1a3dc28281f86715ba94e5e64acb575a54c4 WatchSource:0}: Error finding container 47cbac0ca7e5fdd0511ae4c06edf1a3dc28281f86715ba94e5e64acb575a54c4: Status 404 returned error can't find the container with id 47cbac0ca7e5fdd0511ae4c06edf1a3dc28281f86715ba94e5e64acb575a54c4 Jul 27 23:37:47.749401 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:47.749337 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6"] Jul 27 23:37:47.754704 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:47.754680 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeae3162_3781_4313_ac77_1d847639159b.slice/crio-9b296390a6ca16670552477834706a1928d1d2d4889c99f15d0e5697fc772455 WatchSource:0}: Error finding container 9b296390a6ca16670552477834706a1928d1d2d4889c99f15d0e5697fc772455: Status 404 returned error can't find the container with id 9b296390a6ca16670552477834706a1928d1d2d4889c99f15d0e5697fc772455 Jul 27 23:37:48.231622 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.230912 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-q2k8p"] Jul 27 23:37:48.237636 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.236615 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-q2k8p" Jul 27 23:37:48.241383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.240418 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 27 23:37:48.241383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.240650 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-7sgsp\"" Jul 27 23:37:48.241383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.240877 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 27 23:37:48.245940 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.244148 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-q2k8p"] Jul 27 23:37:48.314372 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.314236 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghhpk\" (UniqueName: \"kubernetes.io/projected/3326b6a4-a622-4c00-b67c-5cf2099775e1-kube-api-access-ghhpk\") pod \"downloads-7c97dc4555-q2k8p\" (UID: \"3326b6a4-a622-4c00-b67c-5cf2099775e1\") " pod="openshift-console/downloads-7c97dc4555-q2k8p" Jul 27 23:37:48.412486 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.412449 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n4wb" event={"ID":"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c","Type":"ContainerStarted","Data":"5f7ecf5743e61225eeac221d52f11883f53ce79bc38207668e01db9941996f20"} Jul 27 23:37:48.413853 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.413804 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9b8f5" event={"ID":"a94ff868-7338-43d5-b8b0-49ebf8277545","Type":"ContainerStarted","Data":"47cbac0ca7e5fdd0511ae4c06edf1a3dc28281f86715ba94e5e64acb575a54c4"} Jul 27 23:37:48.414729 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.414621 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghhpk\" (UniqueName: \"kubernetes.io/projected/3326b6a4-a622-4c00-b67c-5cf2099775e1-kube-api-access-ghhpk\") pod \"downloads-7c97dc4555-q2k8p\" (UID: \"3326b6a4-a622-4c00-b67c-5cf2099775e1\") " pod="openshift-console/downloads-7c97dc4555-q2k8p" Jul 27 23:37:48.415402 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.415349 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" event={"ID":"feae3162-3781-4313-ac77-1d847639159b","Type":"ContainerStarted","Data":"9b296390a6ca16670552477834706a1928d1d2d4889c99f15d0e5697fc772455"} Jul 27 23:37:48.418012 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.417983 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" event={"ID":"e16b04e5-d4a4-40e5-8479-79295c0c2ae1","Type":"ContainerStarted","Data":"b0824afa46ba5c7c0ef29184a68dfd0980c1ed1e5c26898af21eed1e3d76ab78"} Jul 27 23:37:48.419786 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.419737 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p49sh" event={"ID":"1bbd5224-1723-4c96-a1d3-1a454db4cfa8","Type":"ContainerStarted","Data":"39343018ce6cea4519ade35cb1693117b4af0275b69eee9c2451e7d659c7d0a5"} Jul 27 23:37:48.422825 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.422743 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" event={"ID":"960554c4-9d24-4732-bafa-3d829e451e2a","Type":"ContainerStarted","Data":"d07e769950cb95c90ae6bb0605a0259551ec2649103a9729d539c380b5bba3d2"} Jul 27 23:37:48.422825 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.422785 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" event={"ID":"960554c4-9d24-4732-bafa-3d829e451e2a","Type":"ContainerStarted","Data":"325a3a9610418364111311ce21c9deeb8972436c6a557cec33c6d7fca3b7ee27"} Jul 27 23:37:48.424941 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.424482 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-25t97" event={"ID":"d81e8510-04d0-4c8e-8b56-281f0ccdd165","Type":"ContainerStarted","Data":"21e4d5d225f165e4e92d5ab8c3f485dd9b913fe36b3d49109d8e7cf34dcf7dfd"} Jul 27 23:37:48.424941 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.424896 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghhpk\" (UniqueName: \"kubernetes.io/projected/3326b6a4-a622-4c00-b67c-5cf2099775e1-kube-api-access-ghhpk\") pod \"downloads-7c97dc4555-q2k8p\" (UID: \"3326b6a4-a622-4c00-b67c-5cf2099775e1\") " pod="openshift-console/downloads-7c97dc4555-q2k8p" Jul 27 23:37:48.426960 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.426898 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4xtr" event={"ID":"fac2160a-9b05-4bfc-8e98-6205cbf4d252","Type":"ContainerStarted","Data":"df24a86d2e35bed5da1d5ca0ad8e1b74c8ad24ca0afd45a1397fc070e696cc93"} Jul 27 23:37:48.426960 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.426937 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-d4xtr" event={"ID":"fac2160a-9b05-4bfc-8e98-6205cbf4d252","Type":"ContainerStarted","Data":"376bc0859ac12e8ff43716c2954f321a5fda09da9a198495a70bfdaf4bc80a08"} Jul 27 23:37:48.427185 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.426987 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:37:48.428848 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.428820 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" event={"ID":"5d07c216-2c34-40b1-be08-bf1ae2201f45","Type":"ContainerStarted","Data":"e41c9111366450d10d25091aeb3fbe28b4b811fda41627e93cf80911de32f8f9"} Jul 27 23:37:48.431085 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.430997 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" event={"ID":"2714f10c-b53e-4923-be6f-941f9f8d060d","Type":"ContainerStarted","Data":"11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863"} Jul 27 23:37:48.431085 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.431039 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" event={"ID":"2714f10c-b53e-4923-be6f-941f9f8d060d","Type":"ContainerStarted","Data":"ed5fc150d518beb4e0f4656e90125580c4e715ef04a4930da59f2983f00cd669"} Jul 27 23:37:48.431395 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.431355 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:37:48.436629 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.436570 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-64znh" Jul 27 23:37:48.440992 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.440946 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" podStartSLOduration=61.440931707 podStartE2EDuration="1m1.440931707s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:48.440862228 +0000 UTC m=+68.905244366" watchObservedRunningTime="2026-07-27 23:37:48.440931707 +0000 UTC m=+68.905313843" Jul 27 23:37:48.456450 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.456394 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-d4xtr" podStartSLOduration=68.456375599 podStartE2EDuration="1m8.456375599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:48.455906024 +0000 UTC m=+68.920288163" watchObservedRunningTime="2026-07-27 23:37:48.456375599 +0000 UTC m=+68.920757739" Jul 27 23:37:48.502374 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.501533 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" podStartSLOduration=68.501512338 podStartE2EDuration="1m8.501512338s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:48.500204043 +0000 UTC m=+68.964586181" watchObservedRunningTime="2026-07-27 23:37:48.501512338 +0000 UTC m=+68.965894457" Jul 27 23:37:48.555162 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.555131 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-q2k8p" Jul 27 23:37:48.760973 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:48.760854 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-q2k8p"] Jul 27 23:37:48.770504 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:37:48.770463 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3326b6a4_a622_4c00_b67c_5cf2099775e1.slice/crio-e4bdda43fcfb146cf4db8f8bf7ca49737bd996e54cd3d550e6c3c41791ea42c8 WatchSource:0}: Error finding container e4bdda43fcfb146cf4db8f8bf7ca49737bd996e54cd3d550e6c3c41791ea42c8: Status 404 returned error can't find the container with id e4bdda43fcfb146cf4db8f8bf7ca49737bd996e54cd3d550e6c3c41791ea42c8 Jul 27 23:37:49.081190 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:49.081031 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:49.084355 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:49.084135 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:49.443088 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:49.442525 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-q2k8p" event={"ID":"3326b6a4-a622-4c00-b67c-5cf2099775e1","Type":"ContainerStarted","Data":"e4bdda43fcfb146cf4db8f8bf7ca49737bd996e54cd3d550e6c3c41791ea42c8"} Jul 27 23:37:49.443088 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:49.442986 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:49.445208 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:49.444994 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6c94fdb577-j6wm7" Jul 27 23:37:57.485665 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.485590 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" event={"ID":"e16b04e5-d4a4-40e5-8479-79295c0c2ae1","Type":"ContainerStarted","Data":"b16b521623091af1aa0105395f663b92d2877621cb92e28cb00d84ed61f224ce"} Jul 27 23:37:57.488888 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.488806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9b8f5" event={"ID":"a94ff868-7338-43d5-b8b0-49ebf8277545","Type":"ContainerStarted","Data":"61a96bf91e1cad353bbb2aef4caa6c618398f73e49a62138aaf31428e96e1611"} Jul 27 23:37:57.488888 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.488847 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-9b8f5" event={"ID":"a94ff868-7338-43d5-b8b0-49ebf8277545","Type":"ContainerStarted","Data":"3fc28c53e73155e0d1c99febc4c1b01c6b28171957fdc897970bd442e945e078"} Jul 27 23:37:57.493564 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.492940 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" event={"ID":"feae3162-3781-4313-ac77-1d847639159b","Type":"ContainerStarted","Data":"c4e6d4b333a27bb5d4c766e59f1d5404f14900553d003b4616c6a94ade13593c"} Jul 27 23:37:57.497156 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.497125 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p49sh" event={"ID":"1bbd5224-1723-4c96-a1d3-1a454db4cfa8","Type":"ContainerStarted","Data":"d11d9d56fae1dd72c40125efb7f03df119cab67373c546007f783b8f97948a66"} Jul 27 23:37:57.497278 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.497184 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p49sh" event={"ID":"1bbd5224-1723-4c96-a1d3-1a454db4cfa8","Type":"ContainerStarted","Data":"55be19011ed9e3f7e20db7709beefcb4a16b10c0f5efd1719a563fb08c6fd71b"} Jul 27 23:37:57.497438 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.497422 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p49sh" Jul 27 23:37:57.499877 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.499780 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-25t97" event={"ID":"d81e8510-04d0-4c8e-8b56-281f0ccdd165","Type":"ContainerStarted","Data":"dd846e6e82423eb73846d5759698e5af32b77e49dd36fc7d15d89c2e57ab1f15"} Jul 27 23:37:57.504872 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.504825 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-vnp66" podStartSLOduration=61.117270757 podStartE2EDuration="1m10.504806013s" totalImagesPullingTime="9.387535256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:57.503083532 +0000 UTC m=+77.967465670" watchObservedRunningTime="2026-07-27 23:37:57.504806013 +0000 UTC m=+77.969188152" Jul 27 23:37:57.507209 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.507183 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" event={"ID":"5d07c216-2c34-40b1-be08-bf1ae2201f45","Type":"ContainerStarted","Data":"f3759924c93b55657775d6398f107c4d8444502cfd54f6c21bb4db8bd07df497"} Jul 27 23:37:57.507315 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.507231 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" event={"ID":"5d07c216-2c34-40b1-be08-bf1ae2201f45","Type":"ContainerStarted","Data":"dc277da4e86599f5092d8e2f15840c3aabc46e2199620c86301f1c875e34cc63"} Jul 27 23:37:57.511991 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.511926 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n4wb" event={"ID":"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c","Type":"ContainerStarted","Data":"4645012314d7cd52719ca5fbefec72a5c90a5f5c2fca5409296dab4e5e84ae99"} Jul 27 23:37:57.541794 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.541735 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-6qxg6" podStartSLOduration=49.25881022 podStartE2EDuration="58.541714928s" totalImagesPullingTime="9.282904708s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:57.521463607 +0000 UTC m=+77.985845745" watchObservedRunningTime="2026-07-27 23:37:57.541714928 +0000 UTC m=+78.006097067" Jul 27 23:37:57.542030 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.542004 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-9b8f5" podStartSLOduration=68.252705172 podStartE2EDuration="1m17.541996613s" totalImagesPullingTime="9.289291441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:57.54025265 +0000 UTC m=+78.004634788" watchObservedRunningTime="2026-07-27 23:37:57.541996613 +0000 UTC m=+78.006378752" Jul 27 23:37:57.558264 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.558121 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p49sh" podStartSLOduration=35.083686577 podStartE2EDuration="44.558101572s" totalImagesPullingTime="9.474414995s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:57.556836651 +0000 UTC m=+78.021218815" watchObservedRunningTime="2026-07-27 23:37:57.558101572 +0000 UTC m=+78.022483711" Jul 27 23:37:57.596334 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.596267 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-m84w4" podStartSLOduration=61.176448873 podStartE2EDuration="1m10.596251211s" totalImagesPullingTime="9.419802338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:36:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:57.595370935 +0000 UTC m=+78.059753074" watchObservedRunningTime="2026-07-27 23:37:57.596251211 +0000 UTC m=+78.060633349" Jul 27 23:37:57.596605 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:37:57.596574 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-25t97" podStartSLOduration=68.127966066 podStartE2EDuration="1m17.596562852s" totalImagesPullingTime="9.468596786s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:36:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:37:57.575268774 +0000 UTC m=+78.039650913" watchObservedRunningTime="2026-07-27 23:37:57.596562852 +0000 UTC m=+78.060944991" Jul 27 23:38:06.012919 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:06.012784 2561 patch_prober.go:28] interesting pod/image-registry-55c85fbf77-4r5jv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 27 23:38:06.012919 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:06.012849 2561 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" podUID="2714f10c-b53e-4923-be6f-941f9f8d060d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 27 23:38:07.521994 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:07.521961 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p49sh" Jul 27 23:38:08.187045 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.187007 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z2b8j"] Jul 27 23:38:08.199312 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.199284 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.201806 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.201779 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 27 23:38:08.201939 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.201885 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7v7md\"" Jul 27 23:38:08.202125 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.202100 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 27 23:38:08.202260 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.202168 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 27 23:38:08.202260 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.202233 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 27 23:38:08.281907 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.281875 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-root\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.281907 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.281908 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-accelerators-collector-config\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.282128 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.281932 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57c1eda3-4d9b-457f-bc5f-394cd04af153-metrics-client-ca\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.282128 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.281982 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-sys\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.282128 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.281998 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.282128 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.282023 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-tls\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.282128 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.282039 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-wtmp\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.282128 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.282083 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jbkk\" (UniqueName: \"kubernetes.io/projected/57c1eda3-4d9b-457f-bc5f-394cd04af153-kube-api-access-6jbkk\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.282128 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.282122 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-textfile\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382778 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382747 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-root\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382778 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382779 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-accelerators-collector-config\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382805 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57c1eda3-4d9b-457f-bc5f-394cd04af153-metrics-client-ca\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-sys\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382853 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382878 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-tls\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382902 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-wtmp\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382851 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-root\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.382982 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jbkk\" (UniqueName: \"kubernetes.io/projected/57c1eda3-4d9b-457f-bc5f-394cd04af153-kube-api-access-6jbkk\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.383366 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382987 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-sys\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.383366 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.382991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-textfile\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.383366 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:38:08.383039 2561 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 27 23:38:08.383366 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:38:08.383126 2561 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-tls podName:57c1eda3-4d9b-457f-bc5f-394cd04af153 nodeName:}" failed. No retries permitted until 2026-07-27 23:38:08.883105 +0000 UTC m=+89.347487141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-tls") pod "node-exporter-z2b8j" (UID: "57c1eda3-4d9b-457f-bc5f-394cd04af153") : secret "node-exporter-tls" not found Jul 27 23:38:08.383366 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.383213 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-wtmp\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.383366 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.383316 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-textfile\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.383641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.383403 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-accelerators-collector-config\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.383641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.383592 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/57c1eda3-4d9b-457f-bc5f-394cd04af153-metrics-client-ca\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.385379 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.385349 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.393257 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.393224 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jbkk\" (UniqueName: \"kubernetes.io/projected/57c1eda3-4d9b-457f-bc5f-394cd04af153-kube-api-access-6jbkk\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.567226 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.567190 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-8n4wb" event={"ID":"34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c","Type":"ContainerStarted","Data":"7802a95d4ac1e10e107e8fa7ed9b4f0df887dffde3ebcfd97ac1dddb0de99922"} Jul 27 23:38:08.568702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.568674 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-q2k8p" event={"ID":"3326b6a4-a622-4c00-b67c-5cf2099775e1","Type":"ContainerStarted","Data":"73a96da72ad53c090e35f4733d0ae7f7cc57be834904ad6ac3ed746b9d19f6c9"} Jul 27 23:38:08.568890 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.568873 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-q2k8p" Jul 27 23:38:08.570204 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.570180 2561 patch_prober.go:28] interesting pod/downloads-7c97dc4555-q2k8p container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused" start-of-body= Jul 27 23:38:08.570280 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.570224 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-7c97dc4555-q2k8p" podUID="3326b6a4-a622-4c00-b67c-5cf2099775e1" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.20:8080/\": dial tcp 10.132.0.20:8080: connect: connection refused" Jul 27 23:38:08.587340 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.587296 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-8n4wb" podStartSLOduration=16.729904581 podStartE2EDuration="42.58728195s" totalImagesPullingTime="25.857377369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-27 23:37:26 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:08.585835711 +0000 UTC m=+89.050217848" watchObservedRunningTime="2026-07-27 23:38:08.58728195 +0000 UTC m=+89.051664087" Jul 27 23:38:08.602037 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.601988 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-q2k8p" podStartSLOduration=0.922762642 podStartE2EDuration="20.60197501s" totalImagesPullingTime="19.679212368s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:37:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:08.60059816 +0000 UTC m=+89.064980298" watchObservedRunningTime="2026-07-27 23:38:08.60197501 +0000 UTC m=+89.066357181" Jul 27 23:38:08.887521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.887478 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-tls\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.890487 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.890459 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/57c1eda3-4d9b-457f-bc5f-394cd04af153-node-exporter-tls\") pod \"node-exporter-z2b8j\" (UID: \"57c1eda3-4d9b-457f-bc5f-394cd04af153\") " pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:08.958951 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.958909 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55c85fbf77-4r5jv"] Jul 27 23:38:08.964863 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:08.964837 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:38:09.108400 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:09.108362 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z2b8j" Jul 27 23:38:09.130221 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:38:09.130170 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57c1eda3_4d9b_457f_bc5f_394cd04af153.slice/crio-f52343740aa7ae590b181b28155047da5d0b735564ff37f3548cd4a1f1b725de WatchSource:0}: Error finding container f52343740aa7ae590b181b28155047da5d0b735564ff37f3548cd4a1f1b725de: Status 404 returned error can't find the container with id f52343740aa7ae590b181b28155047da5d0b735564ff37f3548cd4a1f1b725de Jul 27 23:38:09.575446 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:09.575362 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2b8j" event={"ID":"57c1eda3-4d9b-457f-bc5f-394cd04af153","Type":"ContainerStarted","Data":"f52343740aa7ae590b181b28155047da5d0b735564ff37f3548cd4a1f1b725de"} Jul 27 23:38:09.590770 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:09.590737 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-q2k8p" Jul 27 23:38:11.585723 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:11.585681 2561 generic.go:356] "Generic (PLEG): container finished" podID="57c1eda3-4d9b-457f-bc5f-394cd04af153" containerID="210a7bf20740e3a41351f1b430b99435d5243915fc4b6e155d2c101b145ea340" exitCode=0 Jul 27 23:38:11.586218 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:11.585753 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2b8j" event={"ID":"57c1eda3-4d9b-457f-bc5f-394cd04af153","Type":"ContainerDied","Data":"210a7bf20740e3a41351f1b430b99435d5243915fc4b6e155d2c101b145ea340"} Jul 27 23:38:12.592900 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:12.592853 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2b8j" event={"ID":"57c1eda3-4d9b-457f-bc5f-394cd04af153","Type":"ContainerStarted","Data":"8aa389b39c18055be0d56a490036a41055811506ae053c1ce170527759984c55"} Jul 27 23:38:12.593458 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:12.592927 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z2b8j" event={"ID":"57c1eda3-4d9b-457f-bc5f-394cd04af153","Type":"ContainerStarted","Data":"1c40f048f694ab568b1f74ea9f81be5fc9e355567be8b7547ba75ba09812fed9"} Jul 27 23:38:12.620114 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:12.620034 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z2b8j" podStartSLOduration=2.91582632 podStartE2EDuration="4.620013995s" totalImagesPullingTime="1.596625205s" totalInitContainerRuntime="107.56247ms" isStatefulPod=true podCreationTimestamp="2026-07-27 23:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:38:12.619551515 +0000 UTC m=+93.083933653" watchObservedRunningTime="2026-07-27 23:38:12.620013995 +0000 UTC m=+93.084396134" Jul 27 23:38:19.445273 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:19.445243 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-d4xtr" Jul 27 23:38:29.655840 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:29.655771 2561 generic.go:356] "Generic (PLEG): container finished" podID="11cb99ed-8d79-48e3-9691-fc12bb0385d2" containerID="7873a81070c381025e31ef6964cd39b634cb40dce082526b8e81b5e939e13b64" exitCode=0 Jul 27 23:38:29.655840 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:29.655819 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" event={"ID":"11cb99ed-8d79-48e3-9691-fc12bb0385d2","Type":"ContainerDied","Data":"7873a81070c381025e31ef6964cd39b634cb40dce082526b8e81b5e939e13b64"} Jul 27 23:38:29.656270 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:29.656144 2561 scope.go:117] "RemoveContainer" containerID="7873a81070c381025e31ef6964cd39b634cb40dce082526b8e81b5e939e13b64" Jul 27 23:38:30.661657 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:30.661625 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn" event={"ID":"11cb99ed-8d79-48e3-9691-fc12bb0385d2","Type":"ContainerStarted","Data":"82c4ac861513830d742b3067d6615c1ccb995e69d625fdd23ff8d12e6a881165"} Jul 27 23:38:33.983600 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:33.983564 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" podUID="2714f10c-b53e-4923-be6f-941f9f8d060d" containerName="registry" containerID="cri-o://11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863" gracePeriod=30 Jul 27 23:38:34.255162 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.255142 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:38:34.394357 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394324 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-installation-pull-secrets\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394357 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394359 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-trusted-ca\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394583 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394419 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-certificates\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394583 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394452 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2714f10c-b53e-4923-be6f-941f9f8d060d-ca-trust-extracted\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394583 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394497 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394583 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394528 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-image-registry-private-configuration\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394583 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394568 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbf7n\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-kube-api-access-gbf7n\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394837 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394597 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-bound-sa-token\") pod \"2714f10c-b53e-4923-be6f-941f9f8d060d\" (UID: \"2714f10c-b53e-4923-be6f-941f9f8d060d\") " Jul 27 23:38:34.394966 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.394861 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:34.395118 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.395098 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 27 23:38:34.397177 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.397130 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:38:34.397455 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.397429 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-kube-api-access-gbf7n" (OuterVolumeSpecName: "kube-api-access-gbf7n") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "kube-api-access-gbf7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:38:34.397565 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.397459 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 27 23:38:34.397565 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.397518 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:38:34.397678 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.397564 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:38:34.402794 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.402770 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2714f10c-b53e-4923-be6f-941f9f8d060d-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "2714f10c-b53e-4923-be6f-941f9f8d060d" (UID: "2714f10c-b53e-4923-be6f-941f9f8d060d"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:38:34.495908 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495881 2561 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-certificates\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.495908 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495904 2561 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/2714f10c-b53e-4923-be6f-941f9f8d060d-ca-trust-extracted\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.496082 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495914 2561 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-registry-tls\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.496082 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495925 2561 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-image-registry-private-configuration\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.496082 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495934 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbf7n\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-kube-api-access-gbf7n\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.496082 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495942 2561 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2714f10c-b53e-4923-be6f-941f9f8d060d-bound-sa-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.496082 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495951 2561 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/2714f10c-b53e-4923-be6f-941f9f8d060d-installation-pull-secrets\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.496082 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.495960 2561 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/2714f10c-b53e-4923-be6f-941f9f8d060d-trusted-ca\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:38:34.679402 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.679370 2561 generic.go:356] "Generic (PLEG): container finished" podID="c410436e-064f-47ed-a826-87ae5903ca81" containerID="628a57e4aa0a7cab400c5fd00f91efce553b7f1a6abfe8cc37881ee487f2d907" exitCode=0 Jul 27 23:38:34.679619 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.679447 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qcrwd" event={"ID":"c410436e-064f-47ed-a826-87ae5903ca81","Type":"ContainerDied","Data":"628a57e4aa0a7cab400c5fd00f91efce553b7f1a6abfe8cc37881ee487f2d907"} Jul 27 23:38:34.679853 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.679824 2561 scope.go:117] "RemoveContainer" containerID="628a57e4aa0a7cab400c5fd00f91efce553b7f1a6abfe8cc37881ee487f2d907" Jul 27 23:38:34.680672 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.680646 2561 generic.go:356] "Generic (PLEG): container finished" podID="2714f10c-b53e-4923-be6f-941f9f8d060d" containerID="11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863" exitCode=0 Jul 27 23:38:34.680789 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.680688 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" Jul 27 23:38:34.680789 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.680697 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" event={"ID":"2714f10c-b53e-4923-be6f-941f9f8d060d","Type":"ContainerDied","Data":"11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863"} Jul 27 23:38:34.680789 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.680740 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-55c85fbf77-4r5jv" event={"ID":"2714f10c-b53e-4923-be6f-941f9f8d060d","Type":"ContainerDied","Data":"ed5fc150d518beb4e0f4656e90125580c4e715ef04a4930da59f2983f00cd669"} Jul 27 23:38:34.680789 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.680766 2561 scope.go:117] "RemoveContainer" containerID="11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863" Jul 27 23:38:34.691975 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.691958 2561 scope.go:117] "RemoveContainer" containerID="11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863" Jul 27 23:38:34.692246 ip-10-0-142-203 kubenswrapper[2561]: E0727 23:38:34.692224 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863\": container with ID starting with 11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863 not found: ID does not exist" containerID="11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863" Jul 27 23:38:34.692345 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.692252 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863"} err="failed to get container status \"11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863\": rpc error: code = NotFound desc = could not find container \"11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863\": container with ID starting with 11c1e16f8732f36c06204b44dda610bd1dcdf93fa2cef81dc8ae63400d376863 not found: ID does not exist" Jul 27 23:38:34.709576 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.709552 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-55c85fbf77-4r5jv"] Jul 27 23:38:34.715301 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:34.715281 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-55c85fbf77-4r5jv"] Jul 27 23:38:35.456708 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:35.456638 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c94fdb577-j6wm7_960554c4-9d24-4732-bafa-3d829e451e2a/router/0.log" Jul 27 23:38:35.467405 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:35.467382 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-98c8q_2e9f1d79-4550-4983-8c6c-6677b0f82347/serve-healthcheck-canary/0.log" Jul 27 23:38:35.687467 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:35.687382 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-qcrwd" event={"ID":"c410436e-064f-47ed-a826-87ae5903ca81","Type":"ContainerStarted","Data":"f6c37a345364e2f60afe8dfa21299275a41b6280ad16a2e0e16d7cfc29dc851e"} Jul 27 23:38:35.970585 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:35.970552 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2714f10c-b53e-4923-be6f-941f9f8d060d" path="/var/lib/kubelet/pods/2714f10c-b53e-4923-be6f-941f9f8d060d/volumes" Jul 27 23:38:44.721247 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:44.721202 2561 generic.go:356] "Generic (PLEG): container finished" podID="daab6831-caa2-4b41-9ee7-f8ffc5d6000b" containerID="0b28514f30a7ea47cd92bbe87dbd35f596dde01956b51c96a3f8e4f83a2dd8f8" exitCode=0 Jul 27 23:38:44.721627 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:44.721276 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" event={"ID":"daab6831-caa2-4b41-9ee7-f8ffc5d6000b","Type":"ContainerDied","Data":"0b28514f30a7ea47cd92bbe87dbd35f596dde01956b51c96a3f8e4f83a2dd8f8"} Jul 27 23:38:44.721627 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:44.721590 2561 scope.go:117] "RemoveContainer" containerID="0b28514f30a7ea47cd92bbe87dbd35f596dde01956b51c96a3f8e4f83a2dd8f8" Jul 27 23:38:45.727392 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:38:45.727353 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-gg8vv" event={"ID":"daab6831-caa2-4b41-9ee7-f8ffc5d6000b","Type":"ContainerStarted","Data":"ee6ee22e6deb7d3f26b425753ca56a7cd4b5afb1b56bb584c7dc78ef75a5d71d"} Jul 27 23:39:32.527203 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.527122 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9"] Jul 27 23:39:32.527615 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.527405 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2714f10c-b53e-4923-be6f-941f9f8d060d" containerName="registry" Jul 27 23:39:32.527615 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.527419 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2714f10c-b53e-4923-be6f-941f9f8d060d" containerName="registry" Jul 27 23:39:32.527615 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.527465 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2714f10c-b53e-4923-be6f-941f9f8d060d" containerName="registry" Jul 27 23:39:32.529507 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.529486 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.531929 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.531905 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 27 23:39:32.532025 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.531940 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 27 23:39:32.532025 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.532002 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 27 23:39:32.532175 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.532086 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-bwvqw\"" Jul 27 23:39:32.532175 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.532112 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 27 23:39:32.532277 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.532262 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 27 23:39:32.537191 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.537173 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 27 23:39:32.541460 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.541437 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9"] Jul 27 23:39:32.627524 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627498 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4cmm\" (UniqueName: \"kubernetes.io/projected/69d502fa-5134-4423-b40e-a8b6106fb46d-kube-api-access-t4cmm\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.627524 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627526 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-metrics-client-ca\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.627706 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627544 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-secret-telemeter-client\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.627706 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627568 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-telemeter-client-tls\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.627706 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627627 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-serving-certs-ca-bundle\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.627706 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627689 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-federate-client-tls\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.627853 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627709 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.627853 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.627736 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.728936 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.728896 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-metrics-client-ca\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.728936 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.728938 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-secret-telemeter-client\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729161 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.728981 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-telemeter-client-tls\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729161 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.729030 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-serving-certs-ca-bundle\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729161 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.729090 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-federate-client-tls\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729161 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.729114 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729161 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.729148 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729400 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.729349 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4cmm\" (UniqueName: \"kubernetes.io/projected/69d502fa-5134-4423-b40e-a8b6106fb46d-kube-api-access-t4cmm\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729789 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.729764 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-metrics-client-ca\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.729908 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.729885 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-serving-certs-ca-bundle\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.730160 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.730135 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/69d502fa-5134-4423-b40e-a8b6106fb46d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.731728 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.731709 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.731875 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.731857 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-secret-telemeter-client\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.732032 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.732014 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-federate-client-tls\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.732180 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.732158 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/69d502fa-5134-4423-b40e-a8b6106fb46d-telemeter-client-tls\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.737428 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.737406 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4cmm\" (UniqueName: \"kubernetes.io/projected/69d502fa-5134-4423-b40e-a8b6106fb46d-kube-api-access-t4cmm\") pod \"telemeter-client-6d47dd6f8f-sl4l9\" (UID: \"69d502fa-5134-4423-b40e-a8b6106fb46d\") " pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.839660 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.839570 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" Jul 27 23:39:32.972388 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:32.972366 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9"] Jul 27 23:39:33.899341 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:33.899228 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" event={"ID":"69d502fa-5134-4423-b40e-a8b6106fb46d","Type":"ContainerStarted","Data":"911ddfac955fac6a4bc92bfefeea8eae1445d0fff1d80ec455a17e3383353960"} Jul 27 23:39:35.907685 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:35.907643 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" event={"ID":"69d502fa-5134-4423-b40e-a8b6106fb46d","Type":"ContainerStarted","Data":"74de597c15d2b5d1fe9f1c2c87a5554b3842569a3784c51641fcf5ea1e12fc76"} Jul 27 23:39:36.913425 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:36.913387 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" event={"ID":"69d502fa-5134-4423-b40e-a8b6106fb46d","Type":"ContainerStarted","Data":"6c8599712f6122c6188e9d98ef7fc4220fcfb59f95cf359a986bd5f69031787c"} Jul 27 23:39:36.913906 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:36.913457 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" event={"ID":"69d502fa-5134-4423-b40e-a8b6106fb46d","Type":"ContainerStarted","Data":"e79ebece0f439293b2d74b65087bf0ebfbff22639f04b8309a7e48caf7abc598"} Jul 27 23:39:36.936687 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:39:36.936626 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d47dd6f8f-sl4l9" podStartSLOduration=1.95878032 podStartE2EDuration="4.936609389s" totalImagesPullingTime="2.977829069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:39:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:39:36.935205794 +0000 UTC m=+177.399587933" watchObservedRunningTime="2026-07-27 23:39:36.936609389 +0000 UTC m=+177.400991527" Jul 27 23:40:40.002980 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.002948 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k"] Jul 27 23:40:40.011589 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.011563 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.013670 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.013639 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k"] Jul 27 23:40:40.014200 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.014174 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tcdxf\"" Jul 27 23:40:40.014200 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.014188 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 23:40:40.014398 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.014242 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 23:40:40.125793 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.125765 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.125955 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.125800 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpjnp\" (UniqueName: \"kubernetes.io/projected/ac675393-7304-47a7-ace9-114e5c1ff81e-kube-api-access-wpjnp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.125955 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.125886 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.226683 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.226647 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.226683 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.226679 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpjnp\" (UniqueName: \"kubernetes.io/projected/ac675393-7304-47a7-ace9-114e5c1ff81e-kube-api-access-wpjnp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.226870 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.226706 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.227106 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.227088 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.227106 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.227100 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.235004 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.234971 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpjnp\" (UniqueName: \"kubernetes.io/projected/ac675393-7304-47a7-ace9-114e5c1ff81e-kube-api-access-wpjnp\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.321618 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.321537 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:40.445548 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:40.445517 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k"] Jul 27 23:40:40.448245 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:40:40.448219 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac675393_7304_47a7_ace9_114e5c1ff81e.slice/crio-fd8f6cc31397f286a8df2168ea61a0dd246a8f79c364fca388c79dfccf18c6d1 WatchSource:0}: Error finding container fd8f6cc31397f286a8df2168ea61a0dd246a8f79c364fca388c79dfccf18c6d1: Status 404 returned error can't find the container with id fd8f6cc31397f286a8df2168ea61a0dd246a8f79c364fca388c79dfccf18c6d1 Jul 27 23:40:41.138820 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:41.138778 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" event={"ID":"ac675393-7304-47a7-ace9-114e5c1ff81e","Type":"ContainerStarted","Data":"fd8f6cc31397f286a8df2168ea61a0dd246a8f79c364fca388c79dfccf18c6d1"} Jul 27 23:40:47.163110 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:47.163052 2561 generic.go:356] "Generic (PLEG): container finished" podID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerID="5504a3d615aface12ee81b42d36f1febbddfe64c9ae06f4bb3eeec6f0e91b646" exitCode=0 Jul 27 23:40:47.163531 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:47.163113 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" event={"ID":"ac675393-7304-47a7-ace9-114e5c1ff81e","Type":"ContainerDied","Data":"5504a3d615aface12ee81b42d36f1febbddfe64c9ae06f4bb3eeec6f0e91b646"} Jul 27 23:40:49.172306 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:49.172223 2561 generic.go:356] "Generic (PLEG): container finished" podID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerID="07d7dd90eed0d5c949730fc1646fa4529ccadfda28e2965abe5963fc1bdda7e5" exitCode=0 Jul 27 23:40:49.172645 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:49.172312 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" event={"ID":"ac675393-7304-47a7-ace9-114e5c1ff81e","Type":"ContainerDied","Data":"07d7dd90eed0d5c949730fc1646fa4529ccadfda28e2965abe5963fc1bdda7e5"} Jul 27 23:40:56.200947 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:56.200906 2561 generic.go:356] "Generic (PLEG): container finished" podID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerID="ef8aa61603eb4168c4fa9dadf32c0f5be793d9fd28c1e381587108fdfe9de996" exitCode=0 Jul 27 23:40:56.201512 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:56.200963 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" event={"ID":"ac675393-7304-47a7-ace9-114e5c1ff81e","Type":"ContainerDied","Data":"ef8aa61603eb4168c4fa9dadf32c0f5be793d9fd28c1e381587108fdfe9de996"} Jul 27 23:40:57.331007 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.330983 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:57.363968 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.363941 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-util\") pod \"ac675393-7304-47a7-ace9-114e5c1ff81e\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " Jul 27 23:40:57.364123 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.364001 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-bundle\") pod \"ac675393-7304-47a7-ace9-114e5c1ff81e\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " Jul 27 23:40:57.364123 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.364044 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpjnp\" (UniqueName: \"kubernetes.io/projected/ac675393-7304-47a7-ace9-114e5c1ff81e-kube-api-access-wpjnp\") pod \"ac675393-7304-47a7-ace9-114e5c1ff81e\" (UID: \"ac675393-7304-47a7-ace9-114e5c1ff81e\") " Jul 27 23:40:57.364715 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.364691 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-bundle" (OuterVolumeSpecName: "bundle") pod "ac675393-7304-47a7-ace9-114e5c1ff81e" (UID: "ac675393-7304-47a7-ace9-114e5c1ff81e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:40:57.366194 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.366168 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ac675393-7304-47a7-ace9-114e5c1ff81e-kube-api-access-wpjnp" (OuterVolumeSpecName: "kube-api-access-wpjnp") pod "ac675393-7304-47a7-ace9-114e5c1ff81e" (UID: "ac675393-7304-47a7-ace9-114e5c1ff81e"). InnerVolumeSpecName "kube-api-access-wpjnp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:40:57.371493 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.371468 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-util" (OuterVolumeSpecName: "util") pod "ac675393-7304-47a7-ace9-114e5c1ff81e" (UID: "ac675393-7304-47a7-ace9-114e5c1ff81e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:40:57.464904 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.464843 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-bundle\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:40:57.464904 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.464866 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpjnp\" (UniqueName: \"kubernetes.io/projected/ac675393-7304-47a7-ace9-114e5c1ff81e-kube-api-access-wpjnp\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:40:57.464904 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:57.464876 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ac675393-7304-47a7-ace9-114e5c1ff81e-util\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:40:58.210545 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:58.210513 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" event={"ID":"ac675393-7304-47a7-ace9-114e5c1ff81e","Type":"ContainerDied","Data":"fd8f6cc31397f286a8df2168ea61a0dd246a8f79c364fca388c79dfccf18c6d1"} Jul 27 23:40:58.210730 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:58.210549 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675409672k" Jul 27 23:40:58.210730 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:40:58.210556 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fd8f6cc31397f286a8df2168ea61a0dd246a8f79c364fca388c79dfccf18c6d1" Jul 27 23:41:02.866365 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866286 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5"] Jul 27 23:41:02.866712 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866555 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerName="util" Jul 27 23:41:02.866712 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866565 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerName="util" Jul 27 23:41:02.866712 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866580 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerName="extract" Jul 27 23:41:02.866712 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866586 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerName="extract" Jul 27 23:41:02.866712 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866597 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerName="pull" Jul 27 23:41:02.866712 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866602 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerName="pull" Jul 27 23:41:02.866712 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.866647 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac675393-7304-47a7-ace9-114e5c1ff81e" containerName="extract" Jul 27 23:41:02.895620 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.895586 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5"] Jul 27 23:41:02.895765 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.895678 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:02.897968 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.897942 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-xt7w9\"" Jul 27 23:41:02.898132 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.898077 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:41:02.898132 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:02.898102 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 27 23:41:03.006320 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.006292 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-979bk\" (UniqueName: \"kubernetes.io/projected/08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc-kube-api-access-979bk\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnbm5\" (UID: \"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:03.006486 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.006341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnbm5\" (UID: \"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:03.107704 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.107673 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-979bk\" (UniqueName: \"kubernetes.io/projected/08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc-kube-api-access-979bk\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnbm5\" (UID: \"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:03.107825 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.107720 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnbm5\" (UID: \"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:03.108024 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.108010 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnbm5\" (UID: \"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:03.115441 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.115419 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-979bk\" (UniqueName: \"kubernetes.io/projected/08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc-kube-api-access-979bk\") pod \"cert-manager-operator-controller-manager-55c444bf86-cnbm5\" (UID: \"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:03.205267 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.205210 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" Jul 27 23:41:03.359023 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:03.358993 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5"] Jul 27 23:41:03.362364 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:41:03.362330 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ac4e8c_5a2c_43d9_93c3_0a9a0f7826fc.slice/crio-a27362702f8b99f02dbc9c940ffef043e7d8f4a596f9e70c0d36ac44e5e61e6a WatchSource:0}: Error finding container a27362702f8b99f02dbc9c940ffef043e7d8f4a596f9e70c0d36ac44e5e61e6a: Status 404 returned error can't find the container with id a27362702f8b99f02dbc9c940ffef043e7d8f4a596f9e70c0d36ac44e5e61e6a Jul 27 23:41:04.235565 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:04.235529 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" event={"ID":"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc","Type":"ContainerStarted","Data":"a27362702f8b99f02dbc9c940ffef043e7d8f4a596f9e70c0d36ac44e5e61e6a"} Jul 27 23:41:06.244484 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:06.244455 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" event={"ID":"08ac4e8c-5a2c-43d9-93c3-0a9a0f7826fc","Type":"ContainerStarted","Data":"d00b22f0055d1e00ebf3fc2a33c6310a5bf89940a325cdbbbf2f3677db5486e5"} Jul 27 23:41:06.269661 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:06.269614 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-cnbm5" podStartSLOduration=2.316704388 podStartE2EDuration="4.269598085s" totalImagesPullingTime="1.952893697s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:06.267579678 +0000 UTC m=+266.731961815" watchObservedRunningTime="2026-07-27 23:41:06.269598085 +0000 UTC m=+266.733980226" Jul 27 23:41:07.248516 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.248484 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94"] Jul 27 23:41:07.272582 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.272554 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94"] Jul 27 23:41:07.272747 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.272654 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.275167 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.275141 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 23:41:07.275167 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.275141 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tcdxf\"" Jul 27 23:41:07.275971 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.275950 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 23:41:07.344087 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.344038 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gpfss\" (UniqueName: \"kubernetes.io/projected/910041ae-a9d7-486b-ab24-cfe89c11f05a-kube-api-access-gpfss\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.344217 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.344154 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.344217 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.344183 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.444921 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.444891 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.445073 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.444926 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.445073 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.444954 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gpfss\" (UniqueName: \"kubernetes.io/projected/910041ae-a9d7-486b-ab24-cfe89c11f05a-kube-api-access-gpfss\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.445277 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.445260 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.445321 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.445294 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.453223 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.453198 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gpfss\" (UniqueName: \"kubernetes.io/projected/910041ae-a9d7-486b-ab24-cfe89c11f05a-kube-api-access-gpfss\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.581875 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.581848 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:07.739402 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:07.739371 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94"] Jul 27 23:41:07.742777 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:41:07.742747 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod910041ae_a9d7_486b_ab24_cfe89c11f05a.slice/crio-5fb19922ef5012f7b00b0812e276e712410f0d9670e69a20a5fc6e37488d4175 WatchSource:0}: Error finding container 5fb19922ef5012f7b00b0812e276e712410f0d9670e69a20a5fc6e37488d4175: Status 404 returned error can't find the container with id 5fb19922ef5012f7b00b0812e276e712410f0d9670e69a20a5fc6e37488d4175 Jul 27 23:41:08.254751 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.254717 2561 generic.go:356] "Generic (PLEG): container finished" podID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerID="b081347e9a7f76272745f53de176e5b89cd02a7470732d8e650c7c56d2145d27" exitCode=0 Jul 27 23:41:08.255172 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.254800 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" event={"ID":"910041ae-a9d7-486b-ab24-cfe89c11f05a","Type":"ContainerDied","Data":"b081347e9a7f76272745f53de176e5b89cd02a7470732d8e650c7c56d2145d27"} Jul 27 23:41:08.255172 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.254847 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" event={"ID":"910041ae-a9d7-486b-ab24-cfe89c11f05a","Type":"ContainerStarted","Data":"5fb19922ef5012f7b00b0812e276e712410f0d9670e69a20a5fc6e37488d4175"} Jul 27 23:41:08.766846 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.766808 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8"] Jul 27 23:41:08.783590 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.783565 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8"] Jul 27 23:41:08.783741 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.783668 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:08.786071 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.786025 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-h96zk\"" Jul 27 23:41:08.786275 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.786257 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 27 23:41:08.786335 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.786313 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 27 23:41:08.854573 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.854543 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h5f7\" (UniqueName: \"kubernetes.io/projected/c4c57722-d516-4111-8aa2-e32e3d2eabec-kube-api-access-2h5f7\") pod \"cert-manager-cainjector-559cdd8c87-6zsf8\" (UID: \"c4c57722-d516-4111-8aa2-e32e3d2eabec\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:08.854695 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.854579 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4c57722-d516-4111-8aa2-e32e3d2eabec-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6zsf8\" (UID: \"c4c57722-d516-4111-8aa2-e32e3d2eabec\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:08.955682 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.955652 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h5f7\" (UniqueName: \"kubernetes.io/projected/c4c57722-d516-4111-8aa2-e32e3d2eabec-kube-api-access-2h5f7\") pod \"cert-manager-cainjector-559cdd8c87-6zsf8\" (UID: \"c4c57722-d516-4111-8aa2-e32e3d2eabec\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:08.955800 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.955687 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4c57722-d516-4111-8aa2-e32e3d2eabec-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6zsf8\" (UID: \"c4c57722-d516-4111-8aa2-e32e3d2eabec\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:08.967500 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.967470 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c4c57722-d516-4111-8aa2-e32e3d2eabec-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6zsf8\" (UID: \"c4c57722-d516-4111-8aa2-e32e3d2eabec\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:08.967688 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:08.967670 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h5f7\" (UniqueName: \"kubernetes.io/projected/c4c57722-d516-4111-8aa2-e32e3d2eabec-kube-api-access-2h5f7\") pod \"cert-manager-cainjector-559cdd8c87-6zsf8\" (UID: \"c4c57722-d516-4111-8aa2-e32e3d2eabec\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:09.105686 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:09.105611 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" Jul 27 23:41:09.241231 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:09.241207 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8"] Jul 27 23:41:09.243405 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:41:09.243375 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4c57722_d516_4111_8aa2_e32e3d2eabec.slice/crio-afa6250a0c1aa8a5a16a886b5958a0993824ff856775bba6833e02782190abf7 WatchSource:0}: Error finding container afa6250a0c1aa8a5a16a886b5958a0993824ff856775bba6833e02782190abf7: Status 404 returned error can't find the container with id afa6250a0c1aa8a5a16a886b5958a0993824ff856775bba6833e02782190abf7 Jul 27 23:41:09.259759 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:09.259726 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" event={"ID":"c4c57722-d516-4111-8aa2-e32e3d2eabec","Type":"ContainerStarted","Data":"afa6250a0c1aa8a5a16a886b5958a0993824ff856775bba6833e02782190abf7"} Jul 27 23:41:12.276576 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:12.276533 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" event={"ID":"c4c57722-d516-4111-8aa2-e32e3d2eabec","Type":"ContainerStarted","Data":"85e6a22f5b9dddf592f04649d4a859908b0eee6c37134845ed80bff2bab384ba"} Jul 27 23:41:12.313821 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:12.313772 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6zsf8" podStartSLOduration=1.8296459280000001 podStartE2EDuration="4.313757906s" totalImagesPullingTime="2.484111978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:12.312459378 +0000 UTC m=+272.776841515" watchObservedRunningTime="2026-07-27 23:41:12.313757906 +0000 UTC m=+272.778140043" Jul 27 23:41:17.259260 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.259234 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4c7pq"] Jul 27 23:41:17.262525 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.262507 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.269210 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.269184 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-x2rps\"" Jul 27 23:41:17.286649 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.286626 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4c7pq"] Jul 27 23:41:17.306315 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.306287 2561 generic.go:356] "Generic (PLEG): container finished" podID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerID="f9fd8bf1885d4ad64852d37348fa4ac4026416c11b84522fd32be51da2dcde51" exitCode=0 Jul 27 23:41:17.306417 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.306331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" event={"ID":"910041ae-a9d7-486b-ab24-cfe89c11f05a","Type":"ContainerDied","Data":"f9fd8bf1885d4ad64852d37348fa4ac4026416c11b84522fd32be51da2dcde51"} Jul 27 23:41:17.327741 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.327715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9tck\" (UniqueName: \"kubernetes.io/projected/df92854b-dd40-4b75-ba9a-a753f56fa3a9-kube-api-access-r9tck\") pod \"cert-manager-55b4d954d5-4c7pq\" (UID: \"df92854b-dd40-4b75-ba9a-a753f56fa3a9\") " pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.327840 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.327751 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df92854b-dd40-4b75-ba9a-a753f56fa3a9-bound-sa-token\") pod \"cert-manager-55b4d954d5-4c7pq\" (UID: \"df92854b-dd40-4b75-ba9a-a753f56fa3a9\") " pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.428689 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.428660 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9tck\" (UniqueName: \"kubernetes.io/projected/df92854b-dd40-4b75-ba9a-a753f56fa3a9-kube-api-access-r9tck\") pod \"cert-manager-55b4d954d5-4c7pq\" (UID: \"df92854b-dd40-4b75-ba9a-a753f56fa3a9\") " pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.428821 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.428717 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df92854b-dd40-4b75-ba9a-a753f56fa3a9-bound-sa-token\") pod \"cert-manager-55b4d954d5-4c7pq\" (UID: \"df92854b-dd40-4b75-ba9a-a753f56fa3a9\") " pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.440843 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.440818 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9tck\" (UniqueName: \"kubernetes.io/projected/df92854b-dd40-4b75-ba9a-a753f56fa3a9-kube-api-access-r9tck\") pod \"cert-manager-55b4d954d5-4c7pq\" (UID: \"df92854b-dd40-4b75-ba9a-a753f56fa3a9\") " pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.442164 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.442145 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/df92854b-dd40-4b75-ba9a-a753f56fa3a9-bound-sa-token\") pod \"cert-manager-55b4d954d5-4c7pq\" (UID: \"df92854b-dd40-4b75-ba9a-a753f56fa3a9\") " pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.641182 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.641152 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-4c7pq" Jul 27 23:41:17.776365 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:17.776334 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-4c7pq"] Jul 27 23:41:17.778478 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:41:17.778452 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf92854b_dd40_4b75_ba9a_a753f56fa3a9.slice/crio-e71113ada9644ce9990b4b2fef5e3fb16eba13508a2b9c5a2a3edb8b52620bcd WatchSource:0}: Error finding container e71113ada9644ce9990b4b2fef5e3fb16eba13508a2b9c5a2a3edb8b52620bcd: Status 404 returned error can't find the container with id e71113ada9644ce9990b4b2fef5e3fb16eba13508a2b9c5a2a3edb8b52620bcd Jul 27 23:41:18.313018 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:18.312973 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4c7pq" event={"ID":"df92854b-dd40-4b75-ba9a-a753f56fa3a9","Type":"ContainerStarted","Data":"6521b8ba8ba0326252c43041ac419578efe3113b650baf96f98ed991a124bb07"} Jul 27 23:41:18.313429 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:18.313031 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-4c7pq" event={"ID":"df92854b-dd40-4b75-ba9a-a753f56fa3a9","Type":"ContainerStarted","Data":"e71113ada9644ce9990b4b2fef5e3fb16eba13508a2b9c5a2a3edb8b52620bcd"} Jul 27 23:41:18.314984 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:18.314958 2561 generic.go:356] "Generic (PLEG): container finished" podID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerID="d01a12eb67fed9c5a6c8ee79bd7d4163b4bca4813eaa91c9c68daf6f25fef437" exitCode=0 Jul 27 23:41:18.315113 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:18.315005 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" event={"ID":"910041ae-a9d7-486b-ab24-cfe89c11f05a","Type":"ContainerDied","Data":"d01a12eb67fed9c5a6c8ee79bd7d4163b4bca4813eaa91c9c68daf6f25fef437"} Jul 27 23:41:18.330249 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:18.330199 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-4c7pq" podStartSLOduration=1.330182793 podStartE2EDuration="1.330182793s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:18.329838275 +0000 UTC m=+278.794220412" watchObservedRunningTime="2026-07-27 23:41:18.330182793 +0000 UTC m=+278.794564935" Jul 27 23:41:19.438498 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.438471 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:19.546741 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.546714 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gpfss\" (UniqueName: \"kubernetes.io/projected/910041ae-a9d7-486b-ab24-cfe89c11f05a-kube-api-access-gpfss\") pod \"910041ae-a9d7-486b-ab24-cfe89c11f05a\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " Jul 27 23:41:19.546889 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.546755 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-bundle\") pod \"910041ae-a9d7-486b-ab24-cfe89c11f05a\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " Jul 27 23:41:19.546889 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.546778 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-util\") pod \"910041ae-a9d7-486b-ab24-cfe89c11f05a\" (UID: \"910041ae-a9d7-486b-ab24-cfe89c11f05a\") " Jul 27 23:41:19.547163 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.547132 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-bundle" (OuterVolumeSpecName: "bundle") pod "910041ae-a9d7-486b-ab24-cfe89c11f05a" (UID: "910041ae-a9d7-486b-ab24-cfe89c11f05a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:41:19.548783 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.548763 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/910041ae-a9d7-486b-ab24-cfe89c11f05a-kube-api-access-gpfss" (OuterVolumeSpecName: "kube-api-access-gpfss") pod "910041ae-a9d7-486b-ab24-cfe89c11f05a" (UID: "910041ae-a9d7-486b-ab24-cfe89c11f05a"). InnerVolumeSpecName "kube-api-access-gpfss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:41:19.551135 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.551111 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-util" (OuterVolumeSpecName: "util") pod "910041ae-a9d7-486b-ab24-cfe89c11f05a" (UID: "910041ae-a9d7-486b-ab24-cfe89c11f05a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:41:19.647685 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.647612 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gpfss\" (UniqueName: \"kubernetes.io/projected/910041ae-a9d7-486b-ab24-cfe89c11f05a-kube-api-access-gpfss\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:41:19.647685 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.647646 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-bundle\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:41:19.647685 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:19.647655 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/910041ae-a9d7-486b-ab24-cfe89c11f05a-util\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:41:20.325365 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:20.325326 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" event={"ID":"910041ae-a9d7-486b-ab24-cfe89c11f05a","Type":"ContainerDied","Data":"5fb19922ef5012f7b00b0812e276e712410f0d9670e69a20a5fc6e37488d4175"} Jul 27 23:41:20.325365 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:20.325362 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fdlh94" Jul 27 23:41:20.325563 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:20.325379 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fb19922ef5012f7b00b0812e276e712410f0d9670e69a20a5fc6e37488d4175" Jul 27 23:41:33.028281 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028247 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw"] Jul 27 23:41:33.028632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028532 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerName="util" Jul 27 23:41:33.028632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028544 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerName="util" Jul 27 23:41:33.028632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028551 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerName="pull" Jul 27 23:41:33.028632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028558 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerName="pull" Jul 27 23:41:33.028632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028567 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerName="extract" Jul 27 23:41:33.028632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028572 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerName="extract" Jul 27 23:41:33.028632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.028618 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="910041ae-a9d7-486b-ab24-cfe89c11f05a" containerName="extract" Jul 27 23:41:33.031470 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.031454 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.034306 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.034280 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 27 23:41:33.034595 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.034581 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tcdxf\"" Jul 27 23:41:33.035329 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.035313 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 27 23:41:33.046635 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.046613 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw"] Jul 27 23:41:33.149940 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.149905 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.150117 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.149960 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.150117 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.149996 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgj2k\" (UniqueName: \"kubernetes.io/projected/41b070c4-173c-44ec-b39b-46e40ff1219c-kube-api-access-cgj2k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.250662 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.250634 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.250797 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.250671 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.250797 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.250695 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgj2k\" (UniqueName: \"kubernetes.io/projected/41b070c4-173c-44ec-b39b-46e40ff1219c-kube-api-access-cgj2k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.250994 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.250974 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.251087 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.251045 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.273238 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.273207 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgj2k\" (UniqueName: \"kubernetes.io/projected/41b070c4-173c-44ec-b39b-46e40ff1219c-kube-api-access-cgj2k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.340740 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.340681 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:33.478024 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:33.478002 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw"] Jul 27 23:41:33.480246 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:41:33.480215 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41b070c4_173c_44ec_b39b_46e40ff1219c.slice/crio-dc69ad43ca8503be45fed97b92ab1280167a516a05cce2a21d673545df706738 WatchSource:0}: Error finding container dc69ad43ca8503be45fed97b92ab1280167a516a05cce2a21d673545df706738: Status 404 returned error can't find the container with id dc69ad43ca8503be45fed97b92ab1280167a516a05cce2a21d673545df706738 Jul 27 23:41:34.387235 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:34.387196 2561 generic.go:356] "Generic (PLEG): container finished" podID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerID="9eae25a289532d6fb9bd192a03073bcd7e392213fe594d2b9701f9c221bca98b" exitCode=0 Jul 27 23:41:34.387687 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:34.387255 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" event={"ID":"41b070c4-173c-44ec-b39b-46e40ff1219c","Type":"ContainerDied","Data":"9eae25a289532d6fb9bd192a03073bcd7e392213fe594d2b9701f9c221bca98b"} Jul 27 23:41:34.387687 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:34.387288 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" event={"ID":"41b070c4-173c-44ec-b39b-46e40ff1219c","Type":"ContainerStarted","Data":"dc69ad43ca8503be45fed97b92ab1280167a516a05cce2a21d673545df706738"} Jul 27 23:41:35.393328 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:35.393288 2561 generic.go:356] "Generic (PLEG): container finished" podID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerID="2a1987df5a3e950ad4c33b241d59b9ddb6f726f7b03e3813656f3e3f83000e4e" exitCode=0 Jul 27 23:41:35.393705 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:35.393354 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" event={"ID":"41b070c4-173c-44ec-b39b-46e40ff1219c","Type":"ContainerDied","Data":"2a1987df5a3e950ad4c33b241d59b9ddb6f726f7b03e3813656f3e3f83000e4e"} Jul 27 23:41:36.399308 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:36.399274 2561 generic.go:356] "Generic (PLEG): container finished" podID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerID="34aeec4ac7d72d7e620b38f7064a357cd6d84d03c1c822570508acfa7b0331cf" exitCode=0 Jul 27 23:41:36.399696 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:36.399331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" event={"ID":"41b070c4-173c-44ec-b39b-46e40ff1219c","Type":"ContainerDied","Data":"34aeec4ac7d72d7e620b38f7064a357cd6d84d03c1c822570508acfa7b0331cf"} Jul 27 23:41:37.528711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.528687 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:37.684771 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.684688 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgj2k\" (UniqueName: \"kubernetes.io/projected/41b070c4-173c-44ec-b39b-46e40ff1219c-kube-api-access-cgj2k\") pod \"41b070c4-173c-44ec-b39b-46e40ff1219c\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " Jul 27 23:41:37.684771 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.684751 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-bundle\") pod \"41b070c4-173c-44ec-b39b-46e40ff1219c\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " Jul 27 23:41:37.684991 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.684774 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-util\") pod \"41b070c4-173c-44ec-b39b-46e40ff1219c\" (UID: \"41b070c4-173c-44ec-b39b-46e40ff1219c\") " Jul 27 23:41:37.685504 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.685478 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-bundle" (OuterVolumeSpecName: "bundle") pod "41b070c4-173c-44ec-b39b-46e40ff1219c" (UID: "41b070c4-173c-44ec-b39b-46e40ff1219c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:41:37.686732 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.686710 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41b070c4-173c-44ec-b39b-46e40ff1219c-kube-api-access-cgj2k" (OuterVolumeSpecName: "kube-api-access-cgj2k") pod "41b070c4-173c-44ec-b39b-46e40ff1219c" (UID: "41b070c4-173c-44ec-b39b-46e40ff1219c"). InnerVolumeSpecName "kube-api-access-cgj2k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:41:37.693079 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.693038 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-util" (OuterVolumeSpecName: "util") pod "41b070c4-173c-44ec-b39b-46e40ff1219c" (UID: "41b070c4-173c-44ec-b39b-46e40ff1219c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:41:37.785737 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.785708 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-bundle\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:41:37.785737 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.785734 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/41b070c4-173c-44ec-b39b-46e40ff1219c-util\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:41:37.785893 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:37.785747 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgj2k\" (UniqueName: \"kubernetes.io/projected/41b070c4-173c-44ec-b39b-46e40ff1219c-kube-api-access-cgj2k\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:41:38.409076 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:38.409027 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" event={"ID":"41b070c4-173c-44ec-b39b-46e40ff1219c","Type":"ContainerDied","Data":"dc69ad43ca8503be45fed97b92ab1280167a516a05cce2a21d673545df706738"} Jul 27 23:41:38.409243 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:38.409082 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afbslbw" Jul 27 23:41:38.409243 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:38.409090 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc69ad43ca8503be45fed97b92ab1280167a516a05cce2a21d673545df706738" Jul 27 23:41:39.841306 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:39.841276 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:41:39.841769 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:39.841540 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:41:39.852426 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:39.852406 2561 kubelet.go:1648] "Image garbage collection succeeded" Jul 27 23:41:45.400459 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400422 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd"] Jul 27 23:41:45.402824 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400726 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerName="util" Jul 27 23:41:45.402824 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400737 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerName="util" Jul 27 23:41:45.402824 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400745 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerName="pull" Jul 27 23:41:45.402824 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400750 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerName="pull" Jul 27 23:41:45.402824 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400757 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerName="extract" Jul 27 23:41:45.402824 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400763 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerName="extract" Jul 27 23:41:45.402824 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.400810 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="41b070c4-173c-44ec-b39b-46e40ff1219c" containerName="extract" Jul 27 23:41:45.403598 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.403584 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.406164 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.406143 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 27 23:41:45.406310 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.406289 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 27 23:41:45.406383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.406357 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 27 23:41:45.406383 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.406364 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 27 23:41:45.406485 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.406412 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-xmfgl\"" Jul 27 23:41:45.406608 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.406592 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 27 23:41:45.411979 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.411958 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd"] Jul 27 23:41:45.435220 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.435195 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fe1b081-b396-433f-888b-3e6d28ee0e1c-cert\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.435335 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.435250 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vn5f\" (UniqueName: \"kubernetes.io/projected/9fe1b081-b396-433f-888b-3e6d28ee0e1c-kube-api-access-4vn5f\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.435335 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.435305 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9fe1b081-b396-433f-888b-3e6d28ee0e1c-manager-config\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.435424 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.435369 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fe1b081-b396-433f-888b-3e6d28ee0e1c-metrics-cert\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.535978 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.535942 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fe1b081-b396-433f-888b-3e6d28ee0e1c-metrics-cert\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.536170 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.535991 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fe1b081-b396-433f-888b-3e6d28ee0e1c-cert\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.536170 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.536032 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vn5f\" (UniqueName: \"kubernetes.io/projected/9fe1b081-b396-433f-888b-3e6d28ee0e1c-kube-api-access-4vn5f\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.536170 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.536067 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9fe1b081-b396-433f-888b-3e6d28ee0e1c-manager-config\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.536786 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.536763 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/9fe1b081-b396-433f-888b-3e6d28ee0e1c-manager-config\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.538437 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.538411 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9fe1b081-b396-433f-888b-3e6d28ee0e1c-cert\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.538558 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.538540 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/9fe1b081-b396-433f-888b-3e6d28ee0e1c-metrics-cert\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.544803 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.544784 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vn5f\" (UniqueName: \"kubernetes.io/projected/9fe1b081-b396-433f-888b-3e6d28ee0e1c-kube-api-access-4vn5f\") pod \"lws-controller-manager-65c89cb49-rk5cd\" (UID: \"9fe1b081-b396-433f-888b-3e6d28ee0e1c\") " pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.713710 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.713624 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:45.853077 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.853039 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd"] Jul 27 23:41:45.855284 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:41:45.855257 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fe1b081_b396_433f_888b_3e6d28ee0e1c.slice/crio-8b6e4ad2cea61225afe80a5074c94ba18416b382384a1d61371d5b58cce2e356 WatchSource:0}: Error finding container 8b6e4ad2cea61225afe80a5074c94ba18416b382384a1d61371d5b58cce2e356: Status 404 returned error can't find the container with id 8b6e4ad2cea61225afe80a5074c94ba18416b382384a1d61371d5b58cce2e356 Jul 27 23:41:45.857183 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:45.857168 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 23:41:46.443405 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:46.443357 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" event={"ID":"9fe1b081-b396-433f-888b-3e6d28ee0e1c","Type":"ContainerStarted","Data":"8b6e4ad2cea61225afe80a5074c94ba18416b382384a1d61371d5b58cce2e356"} Jul 27 23:41:47.449443 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:47.449362 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" event={"ID":"9fe1b081-b396-433f-888b-3e6d28ee0e1c","Type":"ContainerStarted","Data":"d1929dd04f2b8022cfc3ce8085d8730f9748bd899b941e684cf498f287bcecc1"} Jul 27 23:41:47.449839 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:47.449493 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:41:47.470527 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:47.470486 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" podStartSLOduration=0.980886479 podStartE2EDuration="2.470471522s" totalImagesPullingTime="1.489585043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:47.468973602 +0000 UTC m=+307.933355740" watchObservedRunningTime="2026-07-27 23:41:47.470471522 +0000 UTC m=+307.934853660" Jul 27 23:41:48.777707 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.777675 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng"] Jul 27 23:41:48.781148 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.781130 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.784372 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.784350 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rhhx7\"" Jul 27 23:41:48.785165 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.785136 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 27 23:41:48.785268 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.785175 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 27 23:41:48.785432 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.785416 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 27 23:41:48.785841 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.785826 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 27 23:41:48.800868 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.800845 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng"] Jul 27 23:41:48.862063 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.862037 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnv5k\" (UniqueName: \"kubernetes.io/projected/d45f0828-7f4b-48ef-a200-217de6272e42-kube-api-access-vnv5k\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.862211 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.862103 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d45f0828-7f4b-48ef-a200-217de6272e42-manifests\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.862211 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.862128 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d45f0828-7f4b-48ef-a200-217de6272e42-tmp\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.862211 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.862151 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d45f0828-7f4b-48ef-a200-217de6272e42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.862324 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.862237 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d45f0828-7f4b-48ef-a200-217de6272e42-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.963388 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.963356 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnv5k\" (UniqueName: \"kubernetes.io/projected/d45f0828-7f4b-48ef-a200-217de6272e42-kube-api-access-vnv5k\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.963551 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.963399 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d45f0828-7f4b-48ef-a200-217de6272e42-manifests\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.963551 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.963415 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d45f0828-7f4b-48ef-a200-217de6272e42-tmp\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.963551 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.963532 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d45f0828-7f4b-48ef-a200-217de6272e42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.963709 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.963577 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d45f0828-7f4b-48ef-a200-217de6272e42-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.963873 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.963843 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/d45f0828-7f4b-48ef-a200-217de6272e42-manifests\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.965662 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.965640 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d45f0828-7f4b-48ef-a200-217de6272e42-tmp\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.965970 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.965944 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/d45f0828-7f4b-48ef-a200-217de6272e42-apiservice-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.966073 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.966037 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d45f0828-7f4b-48ef-a200-217de6272e42-webhook-cert\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:48.971268 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:48.971246 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnv5k\" (UniqueName: \"kubernetes.io/projected/d45f0828-7f4b-48ef-a200-217de6272e42-kube-api-access-vnv5k\") pod \"opendatahub-operator-controller-manager-79b58bb86f-458ng\" (UID: \"d45f0828-7f4b-48ef-a200-217de6272e42\") " pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:49.090914 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:49.090830 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:49.228277 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:49.228154 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng"] Jul 27 23:41:49.230825 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:41:49.230800 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd45f0828_7f4b_48ef_a200_217de6272e42.slice/crio-6b812032ec76cc7765f08680cf1ce7a21fc099bc5152b88151925c06a7a23f1f WatchSource:0}: Error finding container 6b812032ec76cc7765f08680cf1ce7a21fc099bc5152b88151925c06a7a23f1f: Status 404 returned error can't find the container with id 6b812032ec76cc7765f08680cf1ce7a21fc099bc5152b88151925c06a7a23f1f Jul 27 23:41:49.460417 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:49.460332 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" event={"ID":"d45f0828-7f4b-48ef-a200-217de6272e42","Type":"ContainerStarted","Data":"6b812032ec76cc7765f08680cf1ce7a21fc099bc5152b88151925c06a7a23f1f"} Jul 27 23:41:52.475878 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:52.475791 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" event={"ID":"d45f0828-7f4b-48ef-a200-217de6272e42","Type":"ContainerStarted","Data":"9a40f7880bcf30680c482ec5ea6a5029c2bdc24b0400e315a863fac177e30707"} Jul 27 23:41:53.481969 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:53.481885 2561 generic.go:356] "Generic (PLEG): container finished" podID="d45f0828-7f4b-48ef-a200-217de6272e42" containerID="9a40f7880bcf30680c482ec5ea6a5029c2bdc24b0400e315a863fac177e30707" exitCode=0 Jul 27 23:41:53.481969 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:53.481945 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" event={"ID":"d45f0828-7f4b-48ef-a200-217de6272e42","Type":"ContainerDied","Data":"9a40f7880bcf30680c482ec5ea6a5029c2bdc24b0400e315a863fac177e30707"} Jul 27 23:41:54.488213 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:54.488175 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" event={"ID":"d45f0828-7f4b-48ef-a200-217de6272e42","Type":"ContainerStarted","Data":"a2528ad5df59c8e2853e7e7f1d9c64786d7310871ac7c76a2ef835cf885545bf"} Jul 27 23:41:54.488617 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:54.488284 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:41:54.510142 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:54.510086 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" podStartSLOduration=2.281432137 podStartE2EDuration="6.510045647s" totalImagesPullingTime="3.161441275s" totalInitContainerRuntime="1.067172235s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:41:48 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:41:54.508682989 +0000 UTC m=+314.973065123" watchObservedRunningTime="2026-07-27 23:41:54.510045647 +0000 UTC m=+314.974427788" Jul 27 23:41:58.456762 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:41:58.456731 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-65c89cb49-rk5cd" Jul 27 23:42:05.494250 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:05.494219 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-79b58bb86f-458ng" Jul 27 23:42:16.228268 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.228234 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f"] Jul 27 23:42:16.233313 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.233292 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.235877 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.235853 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 27 23:42:16.235988 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.235853 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-ww9gb\"" Jul 27 23:42:16.244769 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.244747 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f"] Jul 27 23:42:16.302418 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.302392 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz"] Jul 27 23:42:16.302826 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.302804 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.302884 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.302843 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.302884 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.302870 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.302963 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.302938 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.303018 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.303004 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.303075 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.303030 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzcfr\" (UniqueName: \"kubernetes.io/projected/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-kube-api-access-jzcfr\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.303123 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.303079 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.303164 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.303133 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.303216 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.303161 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.305932 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.305916 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.308395 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.308369 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 27 23:42:16.308516 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.308370 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 27 23:42:16.308516 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.308451 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-cj4hx\"" Jul 27 23:42:16.308516 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.308503 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 27 23:42:16.316471 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.316454 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz"] Jul 27 23:42:16.404232 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404199 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404232 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404234 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzcfr\" (UniqueName: \"kubernetes.io/projected/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-kube-api-access-jzcfr\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404473 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404263 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.404473 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404289 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404473 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404306 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85dxg\" (UniqueName: \"kubernetes.io/projected/618365d4-2133-4122-a7aa-cc2196ed69f3-kube-api-access-85dxg\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.404473 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404333 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.404473 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404355 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404473 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404405 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404480 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404511 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404536 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404576 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404614 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404688 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404726 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/618365d4-2133-4122-a7aa-cc2196ed69f3-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.404780 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404774 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.405141 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404826 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.405141 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404876 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.405141 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.404944 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.405471 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.405445 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.405595 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.405575 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.406711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.406691 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.406937 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.406922 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.421378 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.421357 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzcfr\" (UniqueName: \"kubernetes.io/projected/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-kube-api-access-jzcfr\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.423805 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.423788 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4e81ccc9-a64d-441c-bf8f-9c8b97eadc61-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f\" (UID: \"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.506271 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.506246 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/618365d4-2133-4122-a7aa-cc2196ed69f3-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.506271 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.506282 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.506506 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.506320 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.506506 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.506346 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85dxg\" (UniqueName: \"kubernetes.io/projected/618365d4-2133-4122-a7aa-cc2196ed69f3-kube-api-access-85dxg\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.506506 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.506365 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.506506 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.506421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.506776 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.506749 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.507162 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.507138 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.508675 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.508648 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.509344 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.509324 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/618365d4-2133-4122-a7aa-cc2196ed69f3-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.509428 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.509396 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.509467 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.509412 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.519273 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.519251 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85dxg\" (UniqueName: \"kubernetes.io/projected/618365d4-2133-4122-a7aa-cc2196ed69f3-kube-api-access-85dxg\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.519406 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.519390 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/618365d4-2133-4122-a7aa-cc2196ed69f3-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-g9fpz\" (UID: \"618365d4-2133-4122-a7aa-cc2196ed69f3\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.543950 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.543909 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:16.616092 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.615811 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:16.690642 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.690614 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f"] Jul 27 23:42:16.693360 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:42:16.693332 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e81ccc9_a64d_441c_bf8f_9c8b97eadc61.slice/crio-46076eccbec94f171edb7d97ce0b04fff95a99bd5510b6612806fdfb01f61045 WatchSource:0}: Error finding container 46076eccbec94f171edb7d97ce0b04fff95a99bd5510b6612806fdfb01f61045: Status 404 returned error can't find the container with id 46076eccbec94f171edb7d97ce0b04fff95a99bd5510b6612806fdfb01f61045 Jul 27 23:42:16.756345 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:16.756301 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz"] Jul 27 23:42:16.758449 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:42:16.758422 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod618365d4_2133_4122_a7aa_cc2196ed69f3.slice/crio-407edd06129e0e7ac5c30f905ae0385a34c496457110b8a00ea58c024843456f WatchSource:0}: Error finding container 407edd06129e0e7ac5c30f905ae0385a34c496457110b8a00ea58c024843456f: Status 404 returned error can't find the container with id 407edd06129e0e7ac5c30f905ae0385a34c496457110b8a00ea58c024843456f Jul 27 23:42:17.597535 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:17.597497 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" event={"ID":"618365d4-2133-4122-a7aa-cc2196ed69f3","Type":"ContainerStarted","Data":"407edd06129e0e7ac5c30f905ae0385a34c496457110b8a00ea58c024843456f"} Jul 27 23:42:17.600007 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:17.599975 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" event={"ID":"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61","Type":"ContainerStarted","Data":"46076eccbec94f171edb7d97ce0b04fff95a99bd5510b6612806fdfb01f61045"} Jul 27 23:42:20.574310 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:20.574263 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:42:20.574641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:20.574302 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:42:20.574641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:20.574339 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:42:20.574641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:20.574380 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:42:20.574641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:20.574422 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:42:21.626798 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:21.626759 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" event={"ID":"618365d4-2133-4122-a7aa-cc2196ed69f3","Type":"ContainerStarted","Data":"e4e089fe0ef2e0676a06b4232b5a9d876b28b763e7c013834427e0a413d05e26"} Jul 27 23:42:21.627303 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:21.626925 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:21.628381 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:21.628358 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" event={"ID":"4e81ccc9-a64d-441c-bf8f-9c8b97eadc61","Type":"ContainerStarted","Data":"5899444f5225317b816f8688715d377c907406ac0f5ab6eb652c15b464c981d7"} Jul 27 23:42:21.628873 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:21.628853 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" Jul 27 23:42:21.649581 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:21.649535 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-g9fpz" podStartSLOduration=1.836078543 podStartE2EDuration="5.649523895s" totalImagesPullingTime="3.813445352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:42:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:42:21.647247952 +0000 UTC m=+342.111630102" watchObservedRunningTime="2026-07-27 23:42:21.649523895 +0000 UTC m=+342.113906032" Jul 27 23:42:21.686536 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:21.686473 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" podStartSLOduration=1.807959378 podStartE2EDuration="5.686454264s" totalImagesPullingTime="3.878494886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:42:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:42:21.684511782 +0000 UTC m=+342.148893921" watchObservedRunningTime="2026-07-27 23:42:21.686454264 +0000 UTC m=+342.150836402" Jul 27 23:42:22.544171 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:22.544137 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:22.548751 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:22.548727 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:22.634130 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:22.634103 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:42:22.635111 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:42:22.635088 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f" Jul 27 23:43:25.033306 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.033269 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7"] Jul 27 23:43:25.036933 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.036909 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.039251 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.039227 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 27 23:43:25.039382 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.039275 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 27 23:43:25.039382 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.039233 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hwkjm\"" Jul 27 23:43:25.046894 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.046875 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7"] Jul 27 23:43:25.170376 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.170334 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.170551 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.170403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.170551 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.170464 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9gdg\" (UniqueName: \"kubernetes.io/projected/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-kube-api-access-h9gdg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.271334 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.271303 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.271493 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.271342 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9gdg\" (UniqueName: \"kubernetes.io/projected/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-kube-api-access-h9gdg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.271493 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.271389 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.271671 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.271650 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.271734 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.271677 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.279763 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.279741 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9gdg\" (UniqueName: \"kubernetes.io/projected/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-kube-api-access-h9gdg\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.346439 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.346381 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:25.477809 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.477787 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7"] Jul 27 23:43:25.480042 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:43:25.480018 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod745631ec_b4c9_44ea_a2d5_54163c6dd2b8.slice/crio-652400389cc3deecf6c91475494913643af4bbb1d5dee82ec0237188836c29d0 WatchSource:0}: Error finding container 652400389cc3deecf6c91475494913643af4bbb1d5dee82ec0237188836c29d0: Status 404 returned error can't find the container with id 652400389cc3deecf6c91475494913643af4bbb1d5dee82ec0237188836c29d0 Jul 27 23:43:25.631512 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.631449 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz"] Jul 27 23:43:25.634912 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.634896 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.642073 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.642024 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz"] Jul 27 23:43:25.777371 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.777342 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.777553 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.777410 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.777553 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.777461 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-drlg6\" (UniqueName: \"kubernetes.io/projected/8b32f2cd-d210-423c-8266-cafb77073608-kube-api-access-drlg6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.877906 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.877875 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.877906 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.877908 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-drlg6\" (UniqueName: \"kubernetes.io/projected/8b32f2cd-d210-423c-8266-cafb77073608-kube-api-access-drlg6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.878138 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.877940 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.878279 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.878260 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.878349 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.878310 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.885355 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.885297 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-drlg6\" (UniqueName: \"kubernetes.io/projected/8b32f2cd-d210-423c-8266-cafb77073608-kube-api-access-drlg6\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:25.926626 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.926596 2561 generic.go:356] "Generic (PLEG): container finished" podID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerID="c6942f0df5b53d1ca3c0a654eddd085e93d99377b589ba21ae416176a596d548" exitCode=0 Jul 27 23:43:25.926737 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.926677 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" event={"ID":"745631ec-b4c9-44ea-a2d5-54163c6dd2b8","Type":"ContainerDied","Data":"c6942f0df5b53d1ca3c0a654eddd085e93d99377b589ba21ae416176a596d548"} Jul 27 23:43:25.926737 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.926724 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" event={"ID":"745631ec-b4c9-44ea-a2d5-54163c6dd2b8","Type":"ContainerStarted","Data":"652400389cc3deecf6c91475494913643af4bbb1d5dee82ec0237188836c29d0"} Jul 27 23:43:25.944325 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:25.944302 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:26.031442 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.031412 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk"] Jul 27 23:43:26.035398 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.035374 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.041756 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.041730 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk"] Jul 27 23:43:26.074015 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.073994 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz"] Jul 27 23:43:26.075389 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:43:26.075365 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b32f2cd_d210_423c_8266_cafb77073608.slice/crio-ee8c43911f3710e9455e542046219753d5321235c6867464e6411b664b94bc77 WatchSource:0}: Error finding container ee8c43911f3710e9455e542046219753d5321235c6867464e6411b664b94bc77: Status 404 returned error can't find the container with id ee8c43911f3710e9455e542046219753d5321235c6867464e6411b664b94bc77 Jul 27 23:43:26.181701 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.181676 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.181810 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.181710 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cddn\" (UniqueName: \"kubernetes.io/projected/ca799efc-6596-43e5-8a4d-ea414e26c74c-kube-api-access-7cddn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.181810 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.181794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.282779 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.282748 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.282910 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.282811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.282910 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.282841 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cddn\" (UniqueName: \"kubernetes.io/projected/ca799efc-6596-43e5-8a4d-ea414e26c74c-kube-api-access-7cddn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.283229 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.283210 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.283280 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.283215 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.290911 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.290890 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cddn\" (UniqueName: \"kubernetes.io/projected/ca799efc-6596-43e5-8a4d-ea414e26c74c-kube-api-access-7cddn\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.346149 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.346121 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:26.435188 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.435156 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks"] Jul 27 23:43:26.448755 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.448725 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.449000 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.448966 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks"] Jul 27 23:43:26.478976 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.478953 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk"] Jul 27 23:43:26.481281 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:43:26.481255 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca799efc_6596_43e5_8a4d_ea414e26c74c.slice/crio-092c6a403743c5db8bd84ef3f771e7f54bf7060db00f29126aaba11673e74eee WatchSource:0}: Error finding container 092c6a403743c5db8bd84ef3f771e7f54bf7060db00f29126aaba11673e74eee: Status 404 returned error can't find the container with id 092c6a403743c5db8bd84ef3f771e7f54bf7060db00f29126aaba11673e74eee Jul 27 23:43:26.584253 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.584227 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.584368 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.584275 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pzpc7\" (UniqueName: \"kubernetes.io/projected/8020da7b-b59a-4df8-a757-52959e50ba4d-kube-api-access-pzpc7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.584441 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.584379 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.685511 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.685445 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.685511 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.685495 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pzpc7\" (UniqueName: \"kubernetes.io/projected/8020da7b-b59a-4df8-a757-52959e50ba4d-kube-api-access-pzpc7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.685664 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.685535 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.685812 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.685792 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.685858 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.685805 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.693616 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.693593 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pzpc7\" (UniqueName: \"kubernetes.io/projected/8020da7b-b59a-4df8-a757-52959e50ba4d-kube-api-access-pzpc7\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.761975 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.761943 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:26.935694 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.935634 2561 generic.go:356] "Generic (PLEG): container finished" podID="8b32f2cd-d210-423c-8266-cafb77073608" containerID="890229e5746360627bddc36f39f1b0232d25138d94aeb626129d00aea709cc57" exitCode=0 Jul 27 23:43:26.935817 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.935711 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" event={"ID":"8b32f2cd-d210-423c-8266-cafb77073608","Type":"ContainerDied","Data":"890229e5746360627bddc36f39f1b0232d25138d94aeb626129d00aea709cc57"} Jul 27 23:43:26.935817 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.935765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" event={"ID":"8b32f2cd-d210-423c-8266-cafb77073608","Type":"ContainerStarted","Data":"ee8c43911f3710e9455e542046219753d5321235c6867464e6411b664b94bc77"} Jul 27 23:43:26.937354 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.937332 2561 generic.go:356] "Generic (PLEG): container finished" podID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerID="d250f019df27f6343937527393a15c5849e69ed345ab87e2eb4e8627d09b666c" exitCode=0 Jul 27 23:43:26.937452 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.937372 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" event={"ID":"ca799efc-6596-43e5-8a4d-ea414e26c74c","Type":"ContainerDied","Data":"d250f019df27f6343937527393a15c5849e69ed345ab87e2eb4e8627d09b666c"} Jul 27 23:43:26.937452 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.937399 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" event={"ID":"ca799efc-6596-43e5-8a4d-ea414e26c74c","Type":"ContainerStarted","Data":"092c6a403743c5db8bd84ef3f771e7f54bf7060db00f29126aaba11673e74eee"} Jul 27 23:43:26.947278 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:26.947258 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks"] Jul 27 23:43:26.949499 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:43:26.949477 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8020da7b_b59a_4df8_a757_52959e50ba4d.slice/crio-8824eb0801a23f29ef173026dcf259705d25cfe3e16543f19ab0a3051fe39894 WatchSource:0}: Error finding container 8824eb0801a23f29ef173026dcf259705d25cfe3e16543f19ab0a3051fe39894: Status 404 returned error can't find the container with id 8824eb0801a23f29ef173026dcf259705d25cfe3e16543f19ab0a3051fe39894 Jul 27 23:43:27.944428 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:27.944400 2561 generic.go:356] "Generic (PLEG): container finished" podID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerID="baf3bfb9684f5e1f232962d78f3944be0287dbbfb1e668d92c41b61ac6198373" exitCode=0 Jul 27 23:43:27.944794 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:27.944476 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" event={"ID":"ca799efc-6596-43e5-8a4d-ea414e26c74c","Type":"ContainerDied","Data":"baf3bfb9684f5e1f232962d78f3944be0287dbbfb1e668d92c41b61ac6198373"} Jul 27 23:43:27.946439 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:27.946412 2561 generic.go:356] "Generic (PLEG): container finished" podID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerID="bc96cc1c36c9eb6c9dcc10d9cf4dffbcd4aebc4c7f3b832f74f62ea0b82d0546" exitCode=0 Jul 27 23:43:27.946548 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:27.946492 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" event={"ID":"745631ec-b4c9-44ea-a2d5-54163c6dd2b8","Type":"ContainerDied","Data":"bc96cc1c36c9eb6c9dcc10d9cf4dffbcd4aebc4c7f3b832f74f62ea0b82d0546"} Jul 27 23:43:27.948093 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:27.948075 2561 generic.go:356] "Generic (PLEG): container finished" podID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerID="0c7943a54c40b7dca162f96246bda36e39577019b175aa623f8425ab64df5456" exitCode=0 Jul 27 23:43:27.948165 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:27.948097 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" event={"ID":"8020da7b-b59a-4df8-a757-52959e50ba4d","Type":"ContainerDied","Data":"0c7943a54c40b7dca162f96246bda36e39577019b175aa623f8425ab64df5456"} Jul 27 23:43:27.948165 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:27.948131 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" event={"ID":"8020da7b-b59a-4df8-a757-52959e50ba4d","Type":"ContainerStarted","Data":"8824eb0801a23f29ef173026dcf259705d25cfe3e16543f19ab0a3051fe39894"} Jul 27 23:43:28.955359 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.955330 2561 generic.go:356] "Generic (PLEG): container finished" podID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerID="87562e6ba3281ea37363bc4f7a1105209eba243df9db26c4196ff035615c116a" exitCode=0 Jul 27 23:43:28.955716 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.955422 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" event={"ID":"ca799efc-6596-43e5-8a4d-ea414e26c74c","Type":"ContainerDied","Data":"87562e6ba3281ea37363bc4f7a1105209eba243df9db26c4196ff035615c116a"} Jul 27 23:43:28.957335 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.957312 2561 generic.go:356] "Generic (PLEG): container finished" podID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerID="b626512eaf637a2618e91cf9c1bf87b256c8a2b5ee2b6813475df0d24c9c6df6" exitCode=0 Jul 27 23:43:28.957443 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.957386 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" event={"ID":"745631ec-b4c9-44ea-a2d5-54163c6dd2b8","Type":"ContainerDied","Data":"b626512eaf637a2618e91cf9c1bf87b256c8a2b5ee2b6813475df0d24c9c6df6"} Jul 27 23:43:28.959176 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.959157 2561 generic.go:356] "Generic (PLEG): container finished" podID="8b32f2cd-d210-423c-8266-cafb77073608" containerID="2119cfef4c09c11f86a0edc0043db85cde6b27997f70bf8ad977668b0f5e25f0" exitCode=0 Jul 27 23:43:28.959289 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.959235 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" event={"ID":"8b32f2cd-d210-423c-8266-cafb77073608","Type":"ContainerDied","Data":"2119cfef4c09c11f86a0edc0043db85cde6b27997f70bf8ad977668b0f5e25f0"} Jul 27 23:43:28.960993 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.960969 2561 generic.go:356] "Generic (PLEG): container finished" podID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerID="6626ed16391f4359acd9cad23d4d16d8c6acef61538bac501dd51b494e600e7e" exitCode=0 Jul 27 23:43:28.961077 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:28.961030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" event={"ID":"8020da7b-b59a-4df8-a757-52959e50ba4d","Type":"ContainerDied","Data":"6626ed16391f4359acd9cad23d4d16d8c6acef61538bac501dd51b494e600e7e"} Jul 27 23:43:29.973752 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:29.973722 2561 generic.go:356] "Generic (PLEG): container finished" podID="8b32f2cd-d210-423c-8266-cafb77073608" containerID="c6bc6beb3a5190b0e6e2e0730617bdf9ab152a00add45c686f5698eb8fa4fa05" exitCode=0 Jul 27 23:43:29.975040 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:29.975010 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" event={"ID":"8b32f2cd-d210-423c-8266-cafb77073608","Type":"ContainerDied","Data":"c6bc6beb3a5190b0e6e2e0730617bdf9ab152a00add45c686f5698eb8fa4fa05"} Jul 27 23:43:29.975966 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:29.975943 2561 generic.go:356] "Generic (PLEG): container finished" podID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerID="ab903703e1abd0f1b62f4a012ca49d2d27bc31f2bf9134d9e68de4bb49ea8402" exitCode=0 Jul 27 23:43:29.976101 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:29.976006 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" event={"ID":"8020da7b-b59a-4df8-a757-52959e50ba4d","Type":"ContainerDied","Data":"ab903703e1abd0f1b62f4a012ca49d2d27bc31f2bf9134d9e68de4bb49ea8402"} Jul 27 23:43:30.116775 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.116751 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:30.145894 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.145869 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:30.217095 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.217047 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9gdg\" (UniqueName: \"kubernetes.io/projected/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-kube-api-access-h9gdg\") pod \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " Jul 27 23:43:30.217246 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.217163 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-util\") pod \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " Jul 27 23:43:30.217246 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.217198 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-bundle\") pod \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\" (UID: \"745631ec-b4c9-44ea-a2d5-54163c6dd2b8\") " Jul 27 23:43:30.217646 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.217611 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-bundle" (OuterVolumeSpecName: "bundle") pod "745631ec-b4c9-44ea-a2d5-54163c6dd2b8" (UID: "745631ec-b4c9-44ea-a2d5-54163c6dd2b8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:30.219320 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.219300 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-kube-api-access-h9gdg" (OuterVolumeSpecName: "kube-api-access-h9gdg") pod "745631ec-b4c9-44ea-a2d5-54163c6dd2b8" (UID: "745631ec-b4c9-44ea-a2d5-54163c6dd2b8"). InnerVolumeSpecName "kube-api-access-h9gdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:43:30.222542 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.222502 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-util" (OuterVolumeSpecName: "util") pod "745631ec-b4c9-44ea-a2d5-54163c6dd2b8" (UID: "745631ec-b4c9-44ea-a2d5-54163c6dd2b8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:30.318464 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.318436 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-util\") pod \"ca799efc-6596-43e5-8a4d-ea414e26c74c\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " Jul 27 23:43:30.318573 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.318487 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-bundle\") pod \"ca799efc-6596-43e5-8a4d-ea414e26c74c\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " Jul 27 23:43:30.318573 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.318549 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7cddn\" (UniqueName: \"kubernetes.io/projected/ca799efc-6596-43e5-8a4d-ea414e26c74c-kube-api-access-7cddn\") pod \"ca799efc-6596-43e5-8a4d-ea414e26c74c\" (UID: \"ca799efc-6596-43e5-8a4d-ea414e26c74c\") " Jul 27 23:43:30.318697 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.318687 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-util\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:30.318734 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.318699 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-bundle\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:30.318734 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.318708 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9gdg\" (UniqueName: \"kubernetes.io/projected/745631ec-b4c9-44ea-a2d5-54163c6dd2b8-kube-api-access-h9gdg\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:30.319012 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.318987 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-bundle" (OuterVolumeSpecName: "bundle") pod "ca799efc-6596-43e5-8a4d-ea414e26c74c" (UID: "ca799efc-6596-43e5-8a4d-ea414e26c74c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:30.320559 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.320534 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca799efc-6596-43e5-8a4d-ea414e26c74c-kube-api-access-7cddn" (OuterVolumeSpecName: "kube-api-access-7cddn") pod "ca799efc-6596-43e5-8a4d-ea414e26c74c" (UID: "ca799efc-6596-43e5-8a4d-ea414e26c74c"). InnerVolumeSpecName "kube-api-access-7cddn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:43:30.324092 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.324038 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-util" (OuterVolumeSpecName: "util") pod "ca799efc-6596-43e5-8a4d-ea414e26c74c" (UID: "ca799efc-6596-43e5-8a4d-ea414e26c74c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:30.419521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.419484 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-util\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:30.419521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.419512 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca799efc-6596-43e5-8a4d-ea414e26c74c-bundle\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:30.419521 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.419523 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7cddn\" (UniqueName: \"kubernetes.io/projected/ca799efc-6596-43e5-8a4d-ea414e26c74c-kube-api-access-7cddn\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:30.983360 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.983330 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" Jul 27 23:43:30.983826 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.983331 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk" event={"ID":"ca799efc-6596-43e5-8a4d-ea414e26c74c","Type":"ContainerDied","Data":"092c6a403743c5db8bd84ef3f771e7f54bf7060db00f29126aaba11673e74eee"} Jul 27 23:43:30.983826 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.983458 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="092c6a403743c5db8bd84ef3f771e7f54bf7060db00f29126aaba11673e74eee" Jul 27 23:43:30.985025 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.985001 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" event={"ID":"745631ec-b4c9-44ea-a2d5-54163c6dd2b8","Type":"ContainerDied","Data":"652400389cc3deecf6c91475494913643af4bbb1d5dee82ec0237188836c29d0"} Jul 27 23:43:30.985147 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.985043 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="652400389cc3deecf6c91475494913643af4bbb1d5dee82ec0237188836c29d0" Jul 27 23:43:30.985147 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:30.985008 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7" Jul 27 23:43:31.119337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.119308 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:43:31.146092 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.146064 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:31.225677 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.225651 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pzpc7\" (UniqueName: \"kubernetes.io/projected/8020da7b-b59a-4df8-a757-52959e50ba4d-kube-api-access-pzpc7\") pod \"8020da7b-b59a-4df8-a757-52959e50ba4d\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " Jul 27 23:43:31.225816 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.225701 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-util\") pod \"8020da7b-b59a-4df8-a757-52959e50ba4d\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " Jul 27 23:43:31.225858 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.225831 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-bundle\") pod \"8020da7b-b59a-4df8-a757-52959e50ba4d\" (UID: \"8020da7b-b59a-4df8-a757-52959e50ba4d\") " Jul 27 23:43:31.226496 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.226462 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-bundle" (OuterVolumeSpecName: "bundle") pod "8020da7b-b59a-4df8-a757-52959e50ba4d" (UID: "8020da7b-b59a-4df8-a757-52959e50ba4d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:31.227793 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.227775 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8020da7b-b59a-4df8-a757-52959e50ba4d-kube-api-access-pzpc7" (OuterVolumeSpecName: "kube-api-access-pzpc7") pod "8020da7b-b59a-4df8-a757-52959e50ba4d" (UID: "8020da7b-b59a-4df8-a757-52959e50ba4d"). InnerVolumeSpecName "kube-api-access-pzpc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:43:31.230907 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.230876 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-util" (OuterVolumeSpecName: "util") pod "8020da7b-b59a-4df8-a757-52959e50ba4d" (UID: "8020da7b-b59a-4df8-a757-52959e50ba4d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:31.326600 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.326576 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-drlg6\" (UniqueName: \"kubernetes.io/projected/8b32f2cd-d210-423c-8266-cafb77073608-kube-api-access-drlg6\") pod \"8b32f2cd-d210-423c-8266-cafb77073608\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " Jul 27 23:43:31.326726 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.326627 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-util\") pod \"8b32f2cd-d210-423c-8266-cafb77073608\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " Jul 27 23:43:31.326726 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.326655 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-bundle\") pod \"8b32f2cd-d210-423c-8266-cafb77073608\" (UID: \"8b32f2cd-d210-423c-8266-cafb77073608\") " Jul 27 23:43:31.326829 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.326816 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pzpc7\" (UniqueName: \"kubernetes.io/projected/8020da7b-b59a-4df8-a757-52959e50ba4d-kube-api-access-pzpc7\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:31.326885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.326833 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-util\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:31.326885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.326847 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8020da7b-b59a-4df8-a757-52959e50ba4d-bundle\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:31.327242 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.327224 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-bundle" (OuterVolumeSpecName: "bundle") pod "8b32f2cd-d210-423c-8266-cafb77073608" (UID: "8b32f2cd-d210-423c-8266-cafb77073608"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:31.328530 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.328508 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b32f2cd-d210-423c-8266-cafb77073608-kube-api-access-drlg6" (OuterVolumeSpecName: "kube-api-access-drlg6") pod "8b32f2cd-d210-423c-8266-cafb77073608" (UID: "8b32f2cd-d210-423c-8266-cafb77073608"). InnerVolumeSpecName "kube-api-access-drlg6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 27 23:43:31.360180 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.360153 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-util" (OuterVolumeSpecName: "util") pod "8b32f2cd-d210-423c-8266-cafb77073608" (UID: "8b32f2cd-d210-423c-8266-cafb77073608"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 27 23:43:31.427353 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.427328 2561 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-util\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:31.427353 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.427351 2561 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/8b32f2cd-d210-423c-8266-cafb77073608-bundle\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:31.427492 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.427361 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-drlg6\" (UniqueName: \"kubernetes.io/projected/8b32f2cd-d210-423c-8266-cafb77073608-kube-api-access-drlg6\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 27 23:43:31.991168 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.991141 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" event={"ID":"8b32f2cd-d210-423c-8266-cafb77073608","Type":"ContainerDied","Data":"ee8c43911f3710e9455e542046219753d5321235c6867464e6411b664b94bc77"} Jul 27 23:43:31.991522 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.991186 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ee8c43911f3710e9455e542046219753d5321235c6867464e6411b664b94bc77" Jul 27 23:43:31.991522 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.991155 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz" Jul 27 23:43:31.993135 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.993100 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" event={"ID":"8020da7b-b59a-4df8-a757-52959e50ba4d","Type":"ContainerDied","Data":"8824eb0801a23f29ef173026dcf259705d25cfe3e16543f19ab0a3051fe39894"} Jul 27 23:43:31.993237 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.993153 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8824eb0801a23f29ef173026dcf259705d25cfe3e16543f19ab0a3051fe39894" Jul 27 23:43:31.993237 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:43:31.993109 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks" Jul 27 23:44:51.690730 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.690692 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk"] Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691110 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691129 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691143 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691149 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691155 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691160 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691175 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691180 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691185 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b32f2cd-d210-423c-8266-cafb77073608" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691190 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b32f2cd-d210-423c-8266-cafb77073608" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691200 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691208 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691215 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b32f2cd-d210-423c-8266-cafb77073608" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691220 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b32f2cd-d210-423c-8266-cafb77073608" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691228 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691234 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691240 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b32f2cd-d210-423c-8266-cafb77073608" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691245 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b32f2cd-d210-423c-8266-cafb77073608" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691251 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691256 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerName="util" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691261 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691265 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerName="pull" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691271 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691279 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerName="extract" Jul 27 23:44:51.691337 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691343 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca799efc-6596-43e5-8a4d-ea414e26c74c" containerName="extract" Jul 27 23:44:51.692552 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691358 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="745631ec-b4c9-44ea-a2d5-54163c6dd2b8" containerName="extract" Jul 27 23:44:51.692552 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691367 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b32f2cd-d210-423c-8266-cafb77073608" containerName="extract" Jul 27 23:44:51.692552 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.691374 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="8020da7b-b59a-4df8-a757-52959e50ba4d" containerName="extract" Jul 27 23:44:51.694310 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.694293 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.696626 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.696604 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-v8rzj\"" Jul 27 23:44:51.704802 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.704783 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk"] Jul 27 23:44:51.766486 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766455 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766659 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766494 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766659 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766515 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766659 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766561 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0317dd4a-de8a-4245-887f-310a25b7d581-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766659 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766847 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766686 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766847 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766747 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlt4k\" (UniqueName: \"kubernetes.io/projected/0317dd4a-de8a-4245-887f-310a25b7d581-kube-api-access-xlt4k\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766847 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766802 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0317dd4a-de8a-4245-887f-310a25b7d581-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.766847 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.766822 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0317dd4a-de8a-4245-887f-310a25b7d581-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868139 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868098 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0317dd4a-de8a-4245-887f-310a25b7d581-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868139 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868145 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0317dd4a-de8a-4245-887f-310a25b7d581-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868373 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868187 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868373 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868218 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868373 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868360 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868512 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868395 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0317dd4a-de8a-4245-887f-310a25b7d581-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868512 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868424 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868512 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868474 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868512 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868509 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlt4k\" (UniqueName: \"kubernetes.io/projected/0317dd4a-de8a-4245-887f-310a25b7d581-kube-api-access-xlt4k\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868716 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868603 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868803 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868775 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868870 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868786 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.868949 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.868923 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0317dd4a-de8a-4245-887f-310a25b7d581-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.869136 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.869116 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.870552 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.870533 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0317dd4a-de8a-4245-887f-310a25b7d581-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.870885 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.870863 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0317dd4a-de8a-4245-887f-310a25b7d581-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.875944 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.875922 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0317dd4a-de8a-4245-887f-310a25b7d581-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:51.876156 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:51.876137 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlt4k\" (UniqueName: \"kubernetes.io/projected/0317dd4a-de8a-4245-887f-310a25b7d581-kube-api-access-xlt4k\") pod \"maas-default-gateway-openshift-default-78bb78ff78-jgqbk\" (UID: \"0317dd4a-de8a-4245-887f-310a25b7d581\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:52.004702 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.004681 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:52.139722 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.139694 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk"] Jul 27 23:44:52.140563 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:44:52.140538 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0317dd4a_de8a_4245_887f_310a25b7d581.slice/crio-477bb20237a2cb3904fb38be269e70bd828dbcb7898a132e7ecd9461e97abe4c WatchSource:0}: Error finding container 477bb20237a2cb3904fb38be269e70bd828dbcb7898a132e7ecd9461e97abe4c: Status 404 returned error can't find the container with id 477bb20237a2cb3904fb38be269e70bd828dbcb7898a132e7ecd9461e97abe4c Jul 27 23:44:52.142639 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.142609 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:44:52.142711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.142671 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:44:52.142711 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.142704 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:44:52.386503 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.386413 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" event={"ID":"0317dd4a-de8a-4245-887f-310a25b7d581","Type":"ContainerStarted","Data":"6bf1cf4d5be6612a1f500183fc602f46d863080573c69e552d0df4a7466402c5"} Jul 27 23:44:52.386668 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.386522 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" event={"ID":"0317dd4a-de8a-4245-887f-310a25b7d581","Type":"ContainerStarted","Data":"477bb20237a2cb3904fb38be269e70bd828dbcb7898a132e7ecd9461e97abe4c"} Jul 27 23:44:52.408142 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:52.408084 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" podStartSLOduration=1.408031672 podStartE2EDuration="1.408031672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:44:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:44:52.404620792 +0000 UTC m=+492.869002933" watchObservedRunningTime="2026-07-27 23:44:52.408031672 +0000 UTC m=+492.872413813" Jul 27 23:44:53.005266 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:53.005227 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:53.010811 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:53.010780 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:53.391177 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:53.391093 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:44:53.391999 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:44:53.391979 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:46:17.801440 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:17.801412 2561 generic.go:356] "Generic (PLEG): container finished" podID="0317dd4a-de8a-4245-887f-310a25b7d581" containerID="6bf1cf4d5be6612a1f500183fc602f46d863080573c69e552d0df4a7466402c5" exitCode=0 Jul 27 23:46:17.801778 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:17.801461 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" event={"ID":"0317dd4a-de8a-4245-887f-310a25b7d581","Type":"ContainerDied","Data":"6bf1cf4d5be6612a1f500183fc602f46d863080573c69e552d0df4a7466402c5"} Jul 27 23:46:17.801839 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:17.801824 2561 scope.go:117] "RemoveContainer" containerID="6bf1cf4d5be6612a1f500183fc602f46d863080573c69e552d0df4a7466402c5" Jul 27 23:46:17.802805 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:17.802775 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:46:17.802882 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:17.802864 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:46:17.802930 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:17.802918 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 27 23:46:18.808630 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:18.808597 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" event={"ID":"0317dd4a-de8a-4245-887f-310a25b7d581","Type":"ContainerStarted","Data":"e026924edede381483102eb082ec446108ef4484e20b2337f70776fab9a3fc0d"} Jul 27 23:46:19.005390 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:19.005359 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:46:19.006627 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:19.006601 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jgqbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 27 23:46:19.006729 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:19.006650 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" podUID="0317dd4a-de8a-4245-887f-310a25b7d581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 27 23:46:20.005211 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:20.005175 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jgqbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 27 23:46:20.005595 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:20.005226 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" podUID="0317dd4a-de8a-4245-887f-310a25b7d581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 27 23:46:21.005781 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:21.005745 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jgqbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 27 23:46:21.006270 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:21.005817 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" podUID="0317dd4a-de8a-4245-887f-310a25b7d581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 27 23:46:22.005899 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:22.005866 2561 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-jgqbk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body= Jul 27 23:46:22.006299 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:22.005926 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" podUID="0317dd4a-de8a-4245-887f-310a25b7d581" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" Jul 27 23:46:23.008648 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:23.008616 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:46:23.009048 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:23.008938 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:46:23.009526 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:23.009509 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-jgqbk" Jul 27 23:46:25.835351 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.835314 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn"] Jul 27 23:46:25.838950 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.838926 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:25.842290 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.842264 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 27 23:46:25.842406 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.842325 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 27 23:46:25.842406 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.842324 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kmtzt\"" Jul 27 23:46:25.842406 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.842381 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 27 23:46:25.849007 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.848980 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn"] Jul 27 23:46:25.922845 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.922819 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:25.923002 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.922852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9c2b70-de5c-4eea-914e-325a4277d18e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:25.923002 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.922874 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:25.923002 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.922946 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:25.923136 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.923028 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:25.923136 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:25.923122 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwd2m\" (UniqueName: \"kubernetes.io/projected/cd9c2b70-de5c-4eea-914e-325a4277d18e-kube-api-access-mwd2m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024345 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024313 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024522 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024387 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwd2m\" (UniqueName: \"kubernetes.io/projected/cd9c2b70-de5c-4eea-914e-325a4277d18e-kube-api-access-mwd2m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024522 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024410 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024522 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024429 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9c2b70-de5c-4eea-914e-325a4277d18e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024522 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024450 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024735 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024587 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024858 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024836 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024918 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024874 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.024918 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.024904 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.026626 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.026608 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/cd9c2b70-de5c-4eea-914e-325a4277d18e-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.026881 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.026866 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/cd9c2b70-de5c-4eea-914e-325a4277d18e-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.032225 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.032205 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwd2m\" (UniqueName: \"kubernetes.io/projected/cd9c2b70-de5c-4eea-914e-325a4277d18e-kube-api-access-mwd2m\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn\" (UID: \"cd9c2b70-de5c-4eea-914e-325a4277d18e\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.150022 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.149947 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:26.283548 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.283516 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn"] Jul 27 23:46:26.286627 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:46:26.286596 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd9c2b70_de5c_4eea_914e_325a4277d18e.slice/crio-c65e98166ffd657004783eb5009beb941e3cb9b4c639b3aefab5bac54df86981 WatchSource:0}: Error finding container c65e98166ffd657004783eb5009beb941e3cb9b4c639b3aefab5bac54df86981: Status 404 returned error can't find the container with id c65e98166ffd657004783eb5009beb941e3cb9b4c639b3aefab5bac54df86981 Jul 27 23:46:26.849664 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:26.849630 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" event={"ID":"cd9c2b70-de5c-4eea-914e-325a4277d18e","Type":"ContainerStarted","Data":"c65e98166ffd657004783eb5009beb941e3cb9b4c639b3aefab5bac54df86981"} Jul 27 23:46:27.856461 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:27.856430 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" event={"ID":"cd9c2b70-de5c-4eea-914e-325a4277d18e","Type":"ContainerStarted","Data":"210dae28bdb2b7ea87938a5863820a913927c4723af99616f7f4b031684fcefc"} Jul 27 23:46:27.873759 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:27.873713 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" podStartSLOduration=1.796253404 podStartE2EDuration="2.873697007s" totalImagesPullingTime="1.077443603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:46:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:46:27.872707972 +0000 UTC m=+588.337090111" watchObservedRunningTime="2026-07-27 23:46:27.873697007 +0000 UTC m=+588.338079173" Jul 27 23:46:31.238791 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.238758 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4"] Jul 27 23:46:31.242356 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.242335 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.244804 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.244784 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 27 23:46:31.253797 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.253775 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4"] Jul 27 23:46:31.371066 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.371036 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.371224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.371098 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.371224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.371156 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49898aa6-1b9f-4574-8d7e-8d568c59dc72-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.371224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.371176 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.371224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.371192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.371224 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.371215 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtq5z\" (UniqueName: \"kubernetes.io/projected/49898aa6-1b9f-4574-8d7e-8d568c59dc72-kube-api-access-wtq5z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472417 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472380 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472417 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472422 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49898aa6-1b9f-4574-8d7e-8d568c59dc72-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472642 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472441 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472642 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472458 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472642 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472478 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtq5z\" (UniqueName: \"kubernetes.io/projected/49898aa6-1b9f-4574-8d7e-8d568c59dc72-kube-api-access-wtq5z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472642 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472604 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472884 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472860 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.472952 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472865 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.473022 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.472979 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.474692 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.474674 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/49898aa6-1b9f-4574-8d7e-8d568c59dc72-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.475088 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.475049 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/49898aa6-1b9f-4574-8d7e-8d568c59dc72-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.480347 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.480323 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtq5z\" (UniqueName: \"kubernetes.io/projected/49898aa6-1b9f-4574-8d7e-8d568c59dc72-kube-api-access-wtq5z\") pod \"e2e-distinct-simulated-kserve-59975d69f9-vwjr4\" (UID: \"49898aa6-1b9f-4574-8d7e-8d568c59dc72\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.552726 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.552690 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:31.684094 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.684051 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4"] Jul 27 23:46:31.685627 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:46:31.685604 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49898aa6_1b9f_4574_8d7e_8d568c59dc72.slice/crio-eb99295324de83795668db9cb890f1f6ab3c0c0cfe34475ee393735a642ffb53 WatchSource:0}: Error finding container eb99295324de83795668db9cb890f1f6ab3c0c0cfe34475ee393735a642ffb53: Status 404 returned error can't find the container with id eb99295324de83795668db9cb890f1f6ab3c0c0cfe34475ee393735a642ffb53 Jul 27 23:46:31.876655 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:31.876568 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" event={"ID":"49898aa6-1b9f-4574-8d7e-8d568c59dc72","Type":"ContainerStarted","Data":"eb99295324de83795668db9cb890f1f6ab3c0c0cfe34475ee393735a642ffb53"} Jul 27 23:46:32.883094 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:32.883038 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" event={"ID":"49898aa6-1b9f-4574-8d7e-8d568c59dc72","Type":"ContainerStarted","Data":"9b2d0a2e3f6bcf4ff6c5b0f31caf1de6bf5701cde6b8eecaba2fb6d9eaae8c4b"} Jul 27 23:46:32.900964 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:32.900912 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" podStartSLOduration=1.66708983 podStartE2EDuration="1.900895588s" totalImagesPullingTime="233.805758ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:46:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:46:32.899227271 +0000 UTC m=+593.363609411" watchObservedRunningTime="2026-07-27 23:46:32.900895588 +0000 UTC m=+593.365277753" Jul 27 23:46:36.150223 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:36.150186 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:36.150670 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:36.150256 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:36.162753 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:36.162731 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:36.919167 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:36.919131 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn" Jul 27 23:46:39.878917 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:39.878892 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:46:39.880100 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:39.880079 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:46:41.553241 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:41.553210 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:41.553632 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:41.553287 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:41.565677 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:41.565648 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:46:41.943537 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:46:41.943462 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-vwjr4" Jul 27 23:47:10.549391 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.549307 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz"] Jul 27 23:47:10.552665 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.552644 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.555147 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.555122 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 27 23:47:10.566001 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.565979 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz"] Jul 27 23:47:10.620845 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.620819 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4pq9\" (UniqueName: \"kubernetes.io/projected/25f92c31-745a-4bf0-b49b-4961931de18d-kube-api-access-p4pq9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.620976 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.620863 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25f92c31-745a-4bf0-b49b-4961931de18d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.620976 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.620923 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.620976 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.620945 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.621148 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.621034 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.621148 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.621104 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.721804 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.721775 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.721960 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.721811 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.721960 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.721838 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.721960 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.721870 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.721960 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.721912 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4pq9\" (UniqueName: \"kubernetes.io/projected/25f92c31-745a-4bf0-b49b-4961931de18d-kube-api-access-p4pq9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.721960 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.721934 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25f92c31-745a-4bf0-b49b-4961931de18d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.722342 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.722278 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.722342 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.722336 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.722525 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.722388 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.724220 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.724199 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/25f92c31-745a-4bf0-b49b-4961931de18d-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.724454 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.724434 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/25f92c31-745a-4bf0-b49b-4961931de18d-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.731092 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.731037 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4pq9\" (UniqueName: \"kubernetes.io/projected/25f92c31-745a-4bf0-b49b-4961931de18d-kube-api-access-p4pq9\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz\" (UID: \"25f92c31-745a-4bf0-b49b-4961931de18d\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.862021 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.861939 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:10.993422 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.993398 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz"] Jul 27 23:47:10.995236 ip-10-0-142-203 kubenswrapper[2561]: W0727 23:47:10.995211 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25f92c31_745a_4bf0_b49b_4961931de18d.slice/crio-d75df3c839dffccf440894a1e676f6edfc12d62263fc32f6244f997c9241b113 WatchSource:0}: Error finding container d75df3c839dffccf440894a1e676f6edfc12d62263fc32f6244f997c9241b113: Status 404 returned error can't find the container with id d75df3c839dffccf440894a1e676f6edfc12d62263fc32f6244f997c9241b113 Jul 27 23:47:10.996747 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:10.996724 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 27 23:47:11.078096 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:11.078045 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" event={"ID":"25f92c31-745a-4bf0-b49b-4961931de18d","Type":"ContainerStarted","Data":"d75df3c839dffccf440894a1e676f6edfc12d62263fc32f6244f997c9241b113"} Jul 27 23:47:12.084651 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:12.084617 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" event={"ID":"25f92c31-745a-4bf0-b49b-4961931de18d","Type":"ContainerStarted","Data":"fec841ec24786a1971d0d362fc41f2e4f946c5988753f2b539c45463e8c853da"} Jul 27 23:47:12.100682 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:12.100641 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" podStartSLOduration=1.920351431 podStartE2EDuration="2.100625447s" totalImagesPullingTime="180.274016ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-27 23:47:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-27 23:47:12.099043972 +0000 UTC m=+632.563426112" watchObservedRunningTime="2026-07-27 23:47:12.100625447 +0000 UTC m=+632.565007649" Jul 27 23:47:20.862641 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:20.862607 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:20.863159 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:20.862659 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:20.874900 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:20.874876 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:47:21.141418 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:47:21.141331 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz" Jul 27 23:51:39.913005 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:51:39.912976 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:51:39.915754 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:51:39.915732 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:56:39.945272 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:56:39.945241 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 27 23:56:39.949964 ip-10-0-142-203 kubenswrapper[2561]: I0727 23:56:39.949941 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:00:00.239395 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.239360 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62"] Jul 28 00:00:00.243016 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.242997 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" Jul 28 00:00:00.248088 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.248049 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r9jgn\"" Jul 28 00:00:00.254768 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.254745 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62"] Jul 28 00:00:00.277921 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.277901 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" Jul 28 00:00:00.618657 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.618634 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62"] Jul 28 00:00:00.620512 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:00:00.620480 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9658a57c_aa19_4350_922f_df954415fb2f.slice/crio-dd35fd6a920edc9b62387402649a8ffb4ff2f698eb834f433cb171e22f64c54b WatchSource:0}: Error finding container dd35fd6a920edc9b62387402649a8ffb4ff2f698eb834f433cb171e22f64c54b: Status 404 returned error can't find the container with id dd35fd6a920edc9b62387402649a8ffb4ff2f698eb834f433cb171e22f64c54b Jul 28 00:00:00.622889 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.622874 2561 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 28 00:00:00.967892 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:00.967807 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" event={"ID":"9658a57c-aa19-4350-922f-df954415fb2f","Type":"ContainerStarted","Data":"dd35fd6a920edc9b62387402649a8ffb4ff2f698eb834f433cb171e22f64c54b"} Jul 28 00:00:03.986996 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:03.986958 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" event={"ID":"9658a57c-aa19-4350-922f-df954415fb2f","Type":"ContainerStarted","Data":"68f23c5ef9a2742e97ce7c8b895a957a3664edb5be0ad87d7fee7be041f5e0c4"} Jul 28 00:00:04.002430 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:04.002383 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" podStartSLOduration=1.237098221 podStartE2EDuration="4.002368381s" totalImagesPullingTime="2.76527016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:00:04.000212262 +0000 UTC m=+1404.464594423" watchObservedRunningTime="2026-07-28 00:00:04.002368381 +0000 UTC m=+1404.466750576" Jul 28 00:00:04.994662 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:04.994622 2561 generic.go:356] "Generic (PLEG): container finished" podID="9658a57c-aa19-4350-922f-df954415fb2f" containerID="68f23c5ef9a2742e97ce7c8b895a957a3664edb5be0ad87d7fee7be041f5e0c4" exitCode=0 Jul 28 00:00:04.995042 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:04.994690 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" event={"ID":"9658a57c-aa19-4350-922f-df954415fb2f","Type":"ContainerDied","Data":"68f23c5ef9a2742e97ce7c8b895a957a3664edb5be0ad87d7fee7be041f5e0c4"} Jul 28 00:00:06.130590 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:06.130566 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" Jul 28 00:00:07.005779 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:07.005750 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" Jul 28 00:00:07.005779 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:07.005765 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29753280-zct62" event={"ID":"9658a57c-aa19-4350-922f-df954415fb2f","Type":"ContainerDied","Data":"dd35fd6a920edc9b62387402649a8ffb4ff2f698eb834f433cb171e22f64c54b"} Jul 28 00:00:07.006002 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:00:07.005808 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dd35fd6a920edc9b62387402649a8ffb4ff2f698eb834f433cb171e22f64c54b" Jul 28 00:01:39.979080 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:01:39.978963 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:01:39.984235 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:01:39.984210 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:04:10.811303 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.811271 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w"] Jul 28 00:04:10.811772 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.811655 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9658a57c-aa19-4350-922f-df954415fb2f" containerName="cleanup" Jul 28 00:04:10.811772 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.811671 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="9658a57c-aa19-4350-922f-df954415fb2f" containerName="cleanup" Jul 28 00:04:10.811844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.811789 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="9658a57c-aa19-4350-922f-df954415fb2f" containerName="cleanup" Jul 28 00:04:10.814770 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.814753 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.817220 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.817202 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-aa509c8d-openshift-default-dockercfg-5bmth\"" Jul 28 00:04:10.833973 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.833950 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w"] Jul 28 00:04:10.906338 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906307 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-envoy\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906350 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istiod-ca-cert\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906429 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4j5f\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-kube-api-access-z4j5f\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906553 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-podinfo\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906742 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906620 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-socket\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906742 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906647 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-credential-socket\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906742 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-certs\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906742 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906705 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-token\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:10.906742 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:10.906733 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-data\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.007920 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.007886 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-podinfo\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.007929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-socket\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.007946 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-credential-socket\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.007963 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-certs\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.007987 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-token\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008015 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-data\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008085 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-envoy\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008127 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istiod-ca-cert\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008170 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4j5f\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-kube-api-access-z4j5f\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008341 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-socket\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008469 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-certs\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008482 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-credential-socket\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008670 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008508 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-data\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.008917 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.008901 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istiod-ca-cert\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.010381 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.010359 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-envoy\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.010594 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.010577 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-podinfo\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.015173 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.015150 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-token\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.015291 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.015276 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4j5f\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-kube-api-access-z4j5f\") pod \"e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.124596 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.124519 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:11.264102 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.264076 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w"] Jul 28 00:04:11.265508 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:04:11.265476 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12917e2c_3290_4fb6_a3d9_4a854d66e0ef.slice/crio-fd1380a7ca5a55f2326cfc9a9ebceebbf81ca6c0c06644aa211d13b94d0a6d90 WatchSource:0}: Error finding container fd1380a7ca5a55f2326cfc9a9ebceebbf81ca6c0c06644aa211d13b94d0a6d90: Status 404 returned error can't find the container with id fd1380a7ca5a55f2326cfc9a9ebceebbf81ca6c0c06644aa211d13b94d0a6d90 Jul 28 00:04:11.267721 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.267678 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:11.267799 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.267775 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:11.267845 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:11.267817 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:12.275394 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:12.275351 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" event={"ID":"12917e2c-3290-4fb6-a3d9-4a854d66e0ef","Type":"ContainerStarted","Data":"d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7"} Jul 28 00:04:12.275777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:12.275419 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" event={"ID":"12917e2c-3290-4fb6-a3d9-4a854d66e0ef","Type":"ContainerStarted","Data":"fd1380a7ca5a55f2326cfc9a9ebceebbf81ca6c0c06644aa211d13b94d0a6d90"} Jul 28 00:04:12.294951 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:12.294905 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" podStartSLOduration=2.294887175 podStartE2EDuration="2.294887175s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:04:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:04:12.292877291 +0000 UTC m=+1652.757259440" watchObservedRunningTime="2026-07-28 00:04:12.294887175 +0000 UTC m=+1652.759269314" Jul 28 00:04:13.125469 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:13.125435 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:13.130370 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:13.130345 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:13.281107 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:13.281077 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:13.281919 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:13.281902 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:28.537933 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:28.537085 2561 patch_prober.go:28] interesting pod/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:46892->10.132.0.41:15021: read: connection reset by peer" start-of-body= Jul 28 00:04:28.537933 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:28.537151 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": read tcp 10.132.0.2:46892->10.132.0.41:15021: read: connection reset by peer" Jul 28 00:04:29.376555 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:29.376518 2561 generic.go:356] "Generic (PLEG): container finished" podID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerID="d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7" exitCode=0 Jul 28 00:04:29.376724 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:29.376570 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" event={"ID":"12917e2c-3290-4fb6-a3d9-4a854d66e0ef","Type":"ContainerDied","Data":"d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7"} Jul 28 00:04:29.376969 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:29.376952 2561 scope.go:117] "RemoveContainer" containerID="d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7" Jul 28 00:04:29.378014 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:29.377979 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:29.378122 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:29.378102 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:29.378172 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:29.378158 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:30.383187 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:30.383157 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" event={"ID":"12917e2c-3290-4fb6-a3d9-4a854d66e0ef","Type":"ContainerStarted","Data":"b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199"} Jul 28 00:04:31.125462 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:31.125421 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:31.125682 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:31.125659 2561 patch_prober.go:28] interesting pod/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 28 00:04:31.125780 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:31.125751 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 28 00:04:32.124893 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:32.124857 2561 patch_prober.go:28] interesting pod/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 28 00:04:32.125328 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:32.124915 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 28 00:04:33.124896 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:33.124852 2561 patch_prober.go:28] interesting pod/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 28 00:04:33.125339 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:33.124930 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 28 00:04:34.125707 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:34.125666 2561 patch_prober.go:28] interesting pod/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" start-of-body= Jul 28 00:04:34.126133 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:34.125726 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.41:15021/healthz/ready\": dial tcp 10.132.0.41:15021: connect: connection refused" Jul 28 00:04:35.128431 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:35.128399 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:35.128799 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:35.128750 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:35.129337 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:35.129316 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:54.420568 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:54.420536 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w"] Jul 28 00:04:54.421195 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:54.420752 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" containerID="cri-o://b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199" gracePeriod=30 Jul 28 00:04:55.433612 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.433582 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd"] Jul 28 00:04:55.436931 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.436913 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.439433 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.439413 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-af33bc0f-gw-openshift-default-dockercfg-fhrz2\"" Jul 28 00:04:55.455846 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.455805 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd"] Jul 28 00:04:55.521849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.521816 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-socket\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.521849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.521852 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-data\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.522116 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.521888 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-certs\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.522116 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.521912 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-podinfo\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.522116 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.521955 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-envoy\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.522116 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.521986 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgf64\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-kube-api-access-cgf64\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.522116 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.522049 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-credential-socket\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.522116 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.522101 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-token\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.522438 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.522123 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60b42ad-e666-4df1-99dd-af5b2de8608d-istiod-ca-cert\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622601 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622573 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-token\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622762 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622609 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60b42ad-e666-4df1-99dd-af5b2de8608d-istiod-ca-cert\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622762 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622743 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-socket\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622839 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622783 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-data\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622839 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622828 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-certs\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622939 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622851 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-podinfo\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622939 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622905 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-envoy\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.622939 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622929 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgf64\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-kube-api-access-cgf64\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.623111 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.622993 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-credential-socket\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.623172 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.623157 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-socket\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.623339 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.623314 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60b42ad-e666-4df1-99dd-af5b2de8608d-istiod-ca-cert\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.623464 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.623364 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-credential-socket\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.623464 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.623398 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-certs\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.623464 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.623450 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-data\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.625230 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.625209 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-envoy\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.625449 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.625430 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-podinfo\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.630286 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.630266 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-token\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.631028 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.631010 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgf64\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-kube-api-access-cgf64\") pod \"e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.749031 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.748949 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:55.893395 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.893332 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd"] Jul 28 00:04:55.895602 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:04:55.895577 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc60b42ad_e666_4df1_99dd_af5b2de8608d.slice/crio-18ca7071a507164b88214f36be7d712b96ae9136faaefe8354628f81eb878d8d WatchSource:0}: Error finding container 18ca7071a507164b88214f36be7d712b96ae9136faaefe8354628f81eb878d8d: Status 404 returned error can't find the container with id 18ca7071a507164b88214f36be7d712b96ae9136faaefe8354628f81eb878d8d Jul 28 00:04:55.897764 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.897724 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:55.897860 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.897794 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:55.897860 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:55.897823 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:04:56.516444 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:56.516412 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" event={"ID":"c60b42ad-e666-4df1-99dd-af5b2de8608d","Type":"ContainerStarted","Data":"bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a"} Jul 28 00:04:56.516943 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:56.516470 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" event={"ID":"c60b42ad-e666-4df1-99dd-af5b2de8608d","Type":"ContainerStarted","Data":"18ca7071a507164b88214f36be7d712b96ae9136faaefe8354628f81eb878d8d"} Jul 28 00:04:56.536913 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:56.536864 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" podStartSLOduration=1.536847146 podStartE2EDuration="1.536847146s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:04:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:04:56.535714671 +0000 UTC m=+1697.000096811" watchObservedRunningTime="2026-07-28 00:04:56.536847146 +0000 UTC m=+1697.001229284" Jul 28 00:04:56.749987 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:56.749947 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:56.754814 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:56.754787 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:57.522412 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:57.522379 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:57.523468 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:57.523448 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:04:59.672885 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.672863 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:04:59.864577 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864541 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-envoy\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.864577 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864579 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istiod-ca-cert\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.864806 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864613 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-socket\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.864806 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864668 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z4j5f\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-kube-api-access-z4j5f\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.864806 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864715 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-certs\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.864806 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864778 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-data\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.865104 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864826 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-credential-socket\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.865104 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864880 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-podinfo\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.865104 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864928 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-token\") pod \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\" (UID: \"12917e2c-3290-4fb6-a3d9-4a854d66e0ef\") " Jul 28 00:04:59.865104 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.864956 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:04:59.865104 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865085 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:04:59.865375 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865121 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:04:59.865375 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865196 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:04:59.865375 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865302 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.865375 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865322 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.865375 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865340 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.865375 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865358 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.865900 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.865869 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-data" (OuterVolumeSpecName: "istio-data") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:04:59.866933 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.866902 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:04:59.867043 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.866990 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:04:59.867153 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.867120 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-kube-api-access-z4j5f" (OuterVolumeSpecName: "kube-api-access-z4j5f") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "kube-api-access-z4j5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:04:59.867216 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.867199 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-token" (OuterVolumeSpecName: "istio-token") pod "12917e2c-3290-4fb6-a3d9-4a854d66e0ef" (UID: "12917e2c-3290-4fb6-a3d9-4a854d66e0ef"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:04:59.965807 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.965776 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.965807 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.965803 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.965807 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.965813 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.966127 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.965823 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z4j5f\" (UniqueName: \"kubernetes.io/projected/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-kube-api-access-z4j5f\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:04:59.966127 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:04:59.965832 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/12917e2c-3290-4fb6-a3d9-4a854d66e0ef-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:00.538705 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.538670 2561 generic.go:356] "Generic (PLEG): container finished" podID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerID="b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199" exitCode=0 Jul 28 00:05:00.538913 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.538734 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" Jul 28 00:05:00.538913 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.538761 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" event={"ID":"12917e2c-3290-4fb6-a3d9-4a854d66e0ef","Type":"ContainerDied","Data":"b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199"} Jul 28 00:05:00.538913 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.538827 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w" event={"ID":"12917e2c-3290-4fb6-a3d9-4a854d66e0ef","Type":"ContainerDied","Data":"fd1380a7ca5a55f2326cfc9a9ebceebbf81ca6c0c06644aa211d13b94d0a6d90"} Jul 28 00:05:00.538913 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.538857 2561 scope.go:117] "RemoveContainer" containerID="b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199" Jul 28 00:05:00.549916 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.549883 2561 scope.go:117] "RemoveContainer" containerID="d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7" Jul 28 00:05:00.557145 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.557119 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w"] Jul 28 00:05:00.560099 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.560074 2561 scope.go:117] "RemoveContainer" containerID="b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199" Jul 28 00:05:00.560399 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:05:00.560375 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199\": container with ID starting with b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199 not found: ID does not exist" containerID="b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199" Jul 28 00:05:00.560495 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.560411 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199"} err="failed to get container status \"b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199\": rpc error: code = NotFound desc = could not find container \"b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199\": container with ID starting with b51b5c8a94041bf7fbe4b223a044976a0beab4da6c9c2ca185cb66a19c67d199 not found: ID does not exist" Jul 28 00:05:00.560495 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.560436 2561 scope.go:117] "RemoveContainer" containerID="d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7" Jul 28 00:05:00.560753 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:05:00.560729 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7\": container with ID starting with d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7 not found: ID does not exist" containerID="d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7" Jul 28 00:05:00.560818 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.560771 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7"} err="failed to get container status \"d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7\": rpc error: code = NotFound desc = could not find container \"d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7\": container with ID starting with d782c8443b40b99a5ed54aa52a8098a428084361bc237ae4c442c8a32e62c3d7 not found: ID does not exist" Jul 28 00:05:00.561352 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:00.561336 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-aa509c8d-openshift-default-bfff5c886-4sc6w"] Jul 28 00:05:01.972108 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:01.972020 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" path="/var/lib/kubelet/pods/12917e2c-3290-4fb6-a3d9-4a854d66e0ef/volumes" Jul 28 00:05:13.073566 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.072589 2561 patch_prober.go:28] interesting pod/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:37544->10.132.0.42:15021: read: connection reset by peer" start-of-body= Jul 28 00:05:13.073566 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.072664 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": read tcp 10.132.0.2:37544->10.132.0.42:15021: read: connection reset by peer" Jul 28 00:05:13.610220 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.610186 2561 generic.go:356] "Generic (PLEG): container finished" podID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerID="bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a" exitCode=0 Jul 28 00:05:13.610390 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.610235 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" event={"ID":"c60b42ad-e666-4df1-99dd-af5b2de8608d","Type":"ContainerDied","Data":"bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a"} Jul 28 00:05:13.610611 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.610598 2561 scope.go:117] "RemoveContainer" containerID="bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a" Jul 28 00:05:13.611567 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.611540 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:13.611621 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.611606 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:13.611661 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:13.611654 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:14.617257 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:14.617224 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" event={"ID":"c60b42ad-e666-4df1-99dd-af5b2de8608d","Type":"ContainerStarted","Data":"623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698"} Jul 28 00:05:14.749384 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:14.749344 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:05:14.750332 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:14.749985 2561 patch_prober.go:28] interesting pod/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 00:05:14.750332 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:14.750048 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 00:05:15.750256 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:15.750220 2561 patch_prober.go:28] interesting pod/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 00:05:15.750879 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:15.750836 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 00:05:16.749833 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:16.749785 2561 patch_prober.go:28] interesting pod/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 00:05:16.750072 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:16.749857 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 00:05:17.749578 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:17.749538 2561 patch_prober.go:28] interesting pod/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" start-of-body= Jul 28 00:05:17.750209 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:17.750176 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.42:15021/healthz/ready\": dial tcp 10.132.0.42:15021: connect: connection refused" Jul 28 00:05:18.753241 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:18.753210 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:05:18.753611 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:18.753486 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:05:18.754020 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:18.754004 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:05:38.229865 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:38.229825 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd"] Jul 28 00:05:38.230464 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:38.230147 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" containerID="cri-o://623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698" gracePeriod=30 Jul 28 00:05:39.012300 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.012269 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c"] Jul 28 00:05:39.012641 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.012629 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" Jul 28 00:05:39.012683 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.012643 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" Jul 28 00:05:39.012683 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.012660 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" Jul 28 00:05:39.012683 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.012666 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" Jul 28 00:05:39.012780 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.012731 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" Jul 28 00:05:39.015937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.015921 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.018258 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.018236 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-1a3da35c-openshift-default-dockercfg-h2qpp\"" Jul 28 00:05:39.025818 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.025795 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c"] Jul 28 00:05:39.128380 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128341 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-data\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128584 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128421 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gdmfc\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-kube-api-access-gdmfc\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128584 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128470 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-envoy\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128584 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128495 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-podinfo\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128584 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128534 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-socket\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128584 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128559 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-credential-socket\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128584 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128583 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-certs\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128877 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128680 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-token\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.128877 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.128766 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istiod-ca-cert\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.229716 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229679 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-socket\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.229716 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229718 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-credential-socket\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229734 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-certs\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229753 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-token\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229800 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istiod-ca-cert\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229825 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-data\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229887 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gdmfc\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-kube-api-access-gdmfc\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229928 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-envoy\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.229951 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-podinfo\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.230204 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-certs\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.230289 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-credential-socket\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230660 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.230334 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-socket\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230660 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.230360 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-data\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.230660 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.230536 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istiod-ca-cert\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.232300 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.232277 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-envoy\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.232460 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.232444 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-podinfo\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.237655 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.237630 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gdmfc\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-kube-api-access-gdmfc\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.237804 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.237784 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-token\") pod \"e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.328237 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.328149 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:39.459707 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.459685 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c"] Jul 28 00:05:39.461477 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:05:39.461448 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod975f63bc_6b16_4c8c_bbd4_37396eaec1b9.slice/crio-227953cf91c90ba4191a8594b35b9654fd05636396c029b26c24b6875dc83f91 WatchSource:0}: Error finding container 227953cf91c90ba4191a8594b35b9654fd05636396c029b26c24b6875dc83f91: Status 404 returned error can't find the container with id 227953cf91c90ba4191a8594b35b9654fd05636396c029b26c24b6875dc83f91 Jul 28 00:05:39.463872 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.463832 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:39.463968 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.463911 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:39.463968 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.463945 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:39.761160 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.761120 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" event={"ID":"975f63bc-6b16-4c8c-bbd4-37396eaec1b9","Type":"ContainerStarted","Data":"042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972"} Jul 28 00:05:39.761316 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.761184 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" event={"ID":"975f63bc-6b16-4c8c-bbd4-37396eaec1b9","Type":"ContainerStarted","Data":"227953cf91c90ba4191a8594b35b9654fd05636396c029b26c24b6875dc83f91"} Jul 28 00:05:39.779555 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:39.779494 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" podStartSLOduration=1.7794742970000001 podStartE2EDuration="1.779474297s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:05:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:05:39.777789301 +0000 UTC m=+1740.242171431" watchObservedRunningTime="2026-07-28 00:05:39.779474297 +0000 UTC m=+1740.243856436" Jul 28 00:05:40.329276 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:40.329185 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:40.333980 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:40.333957 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:40.766670 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:40.766642 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:40.767516 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:40.767496 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:43.491093 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.491015 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:05:43.672825 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.672780 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-envoy\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.672825 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.672826 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-token\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.672857 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-credential-socket\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.672899 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-podinfo\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.672916 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-certs\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.672954 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60b42ad-e666-4df1-99dd-af5b2de8608d-istiod-ca-cert\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.672980 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-data\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673014 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgf64\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-kube-api-access-cgf64\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673121 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-socket\") pod \"c60b42ad-e666-4df1-99dd-af5b2de8608d\" (UID: \"c60b42ad-e666-4df1-99dd-af5b2de8608d\") " Jul 28 00:05:43.673500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673332 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:05:43.673500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673345 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:05:43.673500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673398 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c60b42ad-e666-4df1-99dd-af5b2de8608d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:05:43.673500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673458 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.673500 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673473 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c60b42ad-e666-4df1-99dd-af5b2de8608d-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.673766 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673501 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.673766 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673570 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:05:43.673997 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.673961 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-data" (OuterVolumeSpecName: "istio-data") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:05:43.675148 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.675120 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-token" (OuterVolumeSpecName: "istio-token") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:05:43.675148 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.675119 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:05:43.675655 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.675630 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:05:43.675740 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.675630 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-kube-api-access-cgf64" (OuterVolumeSpecName: "kube-api-access-cgf64") pod "c60b42ad-e666-4df1-99dd-af5b2de8608d" (UID: "c60b42ad-e666-4df1-99dd-af5b2de8608d"). InnerVolumeSpecName "kube-api-access-cgf64". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:05:43.774967 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.774928 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.774967 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.774961 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.775197 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.774975 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.775197 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.774987 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.775197 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.774997 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgf64\" (UniqueName: \"kubernetes.io/projected/c60b42ad-e666-4df1-99dd-af5b2de8608d-kube-api-access-cgf64\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.775197 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.775006 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c60b42ad-e666-4df1-99dd-af5b2de8608d-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:05:43.782862 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.782833 2561 generic.go:356] "Generic (PLEG): container finished" podID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerID="623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698" exitCode=0 Jul 28 00:05:43.782986 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.782891 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" Jul 28 00:05:43.782986 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.782924 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" event={"ID":"c60b42ad-e666-4df1-99dd-af5b2de8608d","Type":"ContainerDied","Data":"623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698"} Jul 28 00:05:43.782986 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.782984 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd" event={"ID":"c60b42ad-e666-4df1-99dd-af5b2de8608d","Type":"ContainerDied","Data":"18ca7071a507164b88214f36be7d712b96ae9136faaefe8354628f81eb878d8d"} Jul 28 00:05:43.783154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.783007 2561 scope.go:117] "RemoveContainer" containerID="623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698" Jul 28 00:05:43.793166 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.793148 2561 scope.go:117] "RemoveContainer" containerID="bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a" Jul 28 00:05:43.801373 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.801357 2561 scope.go:117] "RemoveContainer" containerID="623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698" Jul 28 00:05:43.801619 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:05:43.801600 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698\": container with ID starting with 623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698 not found: ID does not exist" containerID="623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698" Jul 28 00:05:43.801669 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.801627 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698"} err="failed to get container status \"623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698\": rpc error: code = NotFound desc = could not find container \"623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698\": container with ID starting with 623bcb2c3cb18a9c815221a36f016aeb842aa3e8cb2dacad46da0b7f9f6ec698 not found: ID does not exist" Jul 28 00:05:43.801669 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.801645 2561 scope.go:117] "RemoveContainer" containerID="bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a" Jul 28 00:05:43.801857 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:05:43.801837 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a\": container with ID starting with bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a not found: ID does not exist" containerID="bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a" Jul 28 00:05:43.801894 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.801865 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a"} err="failed to get container status \"bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a\": rpc error: code = NotFound desc = could not find container \"bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a\": container with ID starting with bb8fde3711bafad90074880255942a8b9b3181a4cf79cba9e713dd35a91f7a2a not found: ID does not exist" Jul 28 00:05:43.805154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.805132 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd"] Jul 28 00:05:43.808621 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.808601 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-af33bc0f-gw-openshift-default-8474f5995b-5pqzd"] Jul 28 00:05:43.972254 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:43.972222 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" path="/var/lib/kubelet/pods/c60b42ad-e666-4df1-99dd-af5b2de8608d/volumes" Jul 28 00:05:56.808249 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:56.808206 2561 patch_prober.go:28] interesting pod/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 00:05:56.808661 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:56.808268 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 00:05:57.864797 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:57.864764 2561 generic.go:356] "Generic (PLEG): container finished" podID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerID="042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972" exitCode=0 Jul 28 00:05:57.865266 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:57.864839 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" event={"ID":"975f63bc-6b16-4c8c-bbd4-37396eaec1b9","Type":"ContainerDied","Data":"042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972"} Jul 28 00:05:57.865266 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:57.865260 2561 scope.go:117] "RemoveContainer" containerID="042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972" Jul 28 00:05:57.866318 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:57.866285 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:57.866398 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:57.866371 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:57.866436 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:57.866425 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:05:58.873504 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:58.873469 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" event={"ID":"975f63bc-6b16-4c8c-bbd4-37396eaec1b9","Type":"ContainerStarted","Data":"42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d"} Jul 28 00:05:59.329603 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:59.329564 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:05:59.330028 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:59.329928 2561 patch_prober.go:28] interesting pod/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 28 00:05:59.330254 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:05:59.330229 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 28 00:06:00.329587 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:00.329544 2561 patch_prober.go:28] interesting pod/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 28 00:06:00.330133 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:00.329611 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 28 00:06:01.329371 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:01.329331 2561 patch_prober.go:28] interesting pod/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 28 00:06:01.329680 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:01.329650 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 28 00:06:02.328734 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:02.328685 2561 patch_prober.go:28] interesting pod/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 28 00:06:02.328950 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:02.328754 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 28 00:06:03.332219 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:03.332183 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:06:03.332697 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:03.332575 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:06:03.333333 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:03.333313 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:06:23.259611 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.259571 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c"] Jul 28 00:06:23.260066 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.259838 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" containerID="cri-o://42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d" gracePeriod=30 Jul 28 00:06:23.987838 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.987805 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226"] Jul 28 00:06:23.988340 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.988321 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" Jul 28 00:06:23.988390 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.988344 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" Jul 28 00:06:23.988465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.988455 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" Jul 28 00:06:23.988501 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.988469 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="12917e2c-3290-4fb6-a3d9-4a854d66e0ef" containerName="istio-proxy" Jul 28 00:06:23.988501 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.988481 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" Jul 28 00:06:23.991640 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.991621 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:23.994013 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:23.993995 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-5b6cfa42-openshift-default-dockercfg-npt75\"" Jul 28 00:06:24.002776 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.002750 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226"] Jul 28 00:06:24.153634 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153601 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-data\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.153634 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379918e7-1f8a-4e70-8877-f095845b39d3-istio-podinfo\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.153910 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153661 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-credential-socket\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.153910 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153715 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-certs\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.153910 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153779 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-socket\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.153910 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153806 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-istio-token\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.153910 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153848 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqwvg\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-kube-api-access-xqwvg\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.153910 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153889 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379918e7-1f8a-4e70-8877-f095845b39d3-istiod-ca-cert\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.154196 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.153927 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-envoy\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.254786 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-data\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.254965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254795 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379918e7-1f8a-4e70-8877-f095845b39d3-istio-podinfo\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.254965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254821 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-credential-socket\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.254965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254845 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-certs\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.254965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254889 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-socket\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.254965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254916 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-istio-token\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.254965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254947 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqwvg\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-kube-api-access-xqwvg\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.255319 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.254988 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379918e7-1f8a-4e70-8877-f095845b39d3-istiod-ca-cert\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.255319 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.255020 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-envoy\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.255319 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.255243 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-data\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.255467 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.255381 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-credential-socket\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.255467 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.255453 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-socket\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.255584 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.255492 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-certs\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.255885 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.255860 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379918e7-1f8a-4e70-8877-f095845b39d3-istiod-ca-cert\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.257246 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.257218 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-envoy\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.257367 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.257347 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379918e7-1f8a-4e70-8877-f095845b39d3-istio-podinfo\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.262389 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.262369 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-istio-token\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.262665 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.262478 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqwvg\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-kube-api-access-xqwvg\") pod \"e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.303328 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.303299 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:24.444755 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.444731 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226"] Jul 28 00:06:24.446609 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:06:24.446580 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod379918e7_1f8a_4e70_8877_f095845b39d3.slice/crio-f16f4fea4d30905f281f6ee44cbe9d2e45cca39a2788e5849167fd9883735d63 WatchSource:0}: Error finding container f16f4fea4d30905f281f6ee44cbe9d2e45cca39a2788e5849167fd9883735d63: Status 404 returned error can't find the container with id f16f4fea4d30905f281f6ee44cbe9d2e45cca39a2788e5849167fd9883735d63 Jul 28 00:06:24.448729 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.448695 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:06:24.448830 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.448773 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:06:24.448830 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:24.448819 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:06:25.012469 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:25.012430 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" event={"ID":"379918e7-1f8a-4e70-8877-f095845b39d3","Type":"ContainerStarted","Data":"3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca"} Jul 28 00:06:25.012638 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:25.012488 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" event={"ID":"379918e7-1f8a-4e70-8877-f095845b39d3","Type":"ContainerStarted","Data":"f16f4fea4d30905f281f6ee44cbe9d2e45cca39a2788e5849167fd9883735d63"} Jul 28 00:06:25.035426 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:25.035379 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" podStartSLOduration=2.03536392 podStartE2EDuration="2.03536392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:06:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:06:25.033369155 +0000 UTC m=+1785.497751309" watchObservedRunningTime="2026-07-28 00:06:25.03536392 +0000 UTC m=+1785.499746059" Jul 28 00:06:25.304334 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:25.304256 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:25.309235 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:25.309211 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:26.017522 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:26.017492 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:26.018404 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:26.018384 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:28.508502 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.508477 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:06:28.697255 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697167 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-podinfo\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697255 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697229 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gdmfc\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-kube-api-access-gdmfc\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697460 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697259 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-envoy\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697460 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697312 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-credential-socket\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697460 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697372 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-data\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697460 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697420 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istiod-ca-cert\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697678 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697484 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-socket\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697678 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697510 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-certs\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697678 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697540 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-token\") pod \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\" (UID: \"975f63bc-6b16-4c8c-bbd4-37396eaec1b9\") " Jul 28 00:06:28.697678 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697644 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:06:28.697873 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697834 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:06:28.697987 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.697966 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.698132 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.698115 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.698213 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.698039 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:06:28.698267 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.698228 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:06:28.698432 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.698408 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-data" (OuterVolumeSpecName: "istio-data") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:06:28.699546 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.699523 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-kube-api-access-gdmfc" (OuterVolumeSpecName: "kube-api-access-gdmfc") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "kube-api-access-gdmfc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:06:28.699628 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.699614 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-token" (OuterVolumeSpecName: "istio-token") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:06:28.699820 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.699799 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:06:28.699820 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.699798 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "975f63bc-6b16-4c8c-bbd4-37396eaec1b9" (UID: "975f63bc-6b16-4c8c-bbd4-37396eaec1b9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:06:28.799187 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.799158 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.799187 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.799184 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.799409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.799193 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.799409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.799219 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.799409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.799229 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.799409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.799237 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gdmfc\" (UniqueName: \"kubernetes.io/projected/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-kube-api-access-gdmfc\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:28.799409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:28.799247 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/975f63bc-6b16-4c8c-bbd4-37396eaec1b9-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:06:29.035596 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.035567 2561 generic.go:356] "Generic (PLEG): container finished" podID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerID="42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d" exitCode=0 Jul 28 00:06:29.035767 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.035615 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" Jul 28 00:06:29.035767 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.035621 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" event={"ID":"975f63bc-6b16-4c8c-bbd4-37396eaec1b9","Type":"ContainerDied","Data":"42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d"} Jul 28 00:06:29.035767 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.035725 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c" event={"ID":"975f63bc-6b16-4c8c-bbd4-37396eaec1b9","Type":"ContainerDied","Data":"227953cf91c90ba4191a8594b35b9654fd05636396c029b26c24b6875dc83f91"} Jul 28 00:06:29.035767 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.035748 2561 scope.go:117] "RemoveContainer" containerID="42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d" Jul 28 00:06:29.046177 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.046160 2561 scope.go:117] "RemoveContainer" containerID="042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972" Jul 28 00:06:29.055306 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.054802 2561 scope.go:117] "RemoveContainer" containerID="42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d" Jul 28 00:06:29.055306 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:06:29.055262 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d\": container with ID starting with 42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d not found: ID does not exist" containerID="42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d" Jul 28 00:06:29.055306 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.055297 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d"} err="failed to get container status \"42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d\": rpc error: code = NotFound desc = could not find container \"42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d\": container with ID starting with 42fcfdb1736a7b460de750a509af179843cea9db8d74574f90d056c4ab951d6d not found: ID does not exist" Jul 28 00:06:29.055518 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.055321 2561 scope.go:117] "RemoveContainer" containerID="042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972" Jul 28 00:06:29.055709 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:06:29.055675 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972\": container with ID starting with 042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972 not found: ID does not exist" containerID="042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972" Jul 28 00:06:29.055826 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.055715 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972"} err="failed to get container status \"042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972\": rpc error: code = NotFound desc = could not find container \"042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972\": container with ID starting with 042a24933c196caa15597f28c552e74a1697aaa5f443b0c13d24ec95cfce7972 not found: ID does not exist" Jul 28 00:06:29.057964 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.057945 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c"] Jul 28 00:06:29.061600 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.061580 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-1a3da35c-openshift-default-5554db6bbb-brb9c"] Jul 28 00:06:29.977980 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:29.977944 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" path="/var/lib/kubelet/pods/975f63bc-6b16-4c8c-bbd4-37396eaec1b9/volumes" Jul 28 00:06:40.015636 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:40.015603 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:06:40.022619 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:40.022589 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:06:41.695828 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:41.695562 2561 patch_prober.go:28] interesting pod/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:39592->10.132.0.44:15021: read: connection reset by peer" start-of-body= Jul 28 00:06:41.695828 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:41.695642 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": read tcp 10.132.0.2:39592->10.132.0.44:15021: read: connection reset by peer" Jul 28 00:06:42.105959 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:42.105926 2561 generic.go:356] "Generic (PLEG): container finished" podID="379918e7-1f8a-4e70-8877-f095845b39d3" containerID="3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca" exitCode=0 Jul 28 00:06:42.106163 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:42.105973 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" event={"ID":"379918e7-1f8a-4e70-8877-f095845b39d3","Type":"ContainerDied","Data":"3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca"} Jul 28 00:06:42.106454 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:42.106436 2561 scope.go:117] "RemoveContainer" containerID="3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca" Jul 28 00:06:42.107492 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:42.107463 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:06:42.107550 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:42.107527 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:06:42.107591 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:42.107580 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:06:43.112945 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:43.112912 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" event={"ID":"379918e7-1f8a-4e70-8877-f095845b39d3","Type":"ContainerStarted","Data":"fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8"} Jul 28 00:06:43.304373 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:43.304341 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:43.304579 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:43.304556 2561 patch_prober.go:28] interesting pod/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 28 00:06:43.304629 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:43.304605 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 28 00:06:44.304455 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:44.304425 2561 patch_prober.go:28] interesting pod/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 28 00:06:44.304849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:44.304480 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 28 00:06:45.304383 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:45.304343 2561 patch_prober.go:28] interesting pod/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 28 00:06:45.304575 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:45.304405 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 28 00:06:46.304683 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:46.304647 2561 patch_prober.go:28] interesting pod/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body= Jul 28 00:06:46.305099 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:46.304703 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" Jul 28 00:06:47.307340 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:47.307307 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:47.307733 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:47.307681 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:06:47.308502 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:06:47.308482 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:07:06.483975 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:06.483933 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226"] Jul 28 00:07:06.484630 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:06.484253 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" containerID="cri-o://fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8" gracePeriod=30 Jul 28 00:07:11.741600 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.741575 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:07:11.821225 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821194 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379918e7-1f8a-4e70-8877-f095845b39d3-istiod-ca-cert\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821393 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821235 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-socket\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821393 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821257 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqwvg\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-kube-api-access-xqwvg\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821393 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821285 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-istio-token\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821393 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821307 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-certs\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821393 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821327 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-data\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821393 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821367 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379918e7-1f8a-4e70-8877-f095845b39d3-istio-podinfo\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821713 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821421 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-credential-socket\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821713 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821454 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-envoy\") pod \"379918e7-1f8a-4e70-8877-f095845b39d3\" (UID: \"379918e7-1f8a-4e70-8877-f095845b39d3\") " Jul 28 00:07:11.821713 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821511 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:07:11.821713 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821610 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:07:11.821713 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821699 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/379918e7-1f8a-4e70-8877-f095845b39d3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:07:11.821972 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821779 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/379918e7-1f8a-4e70-8877-f095845b39d3-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.821972 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821796 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.821972 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821809 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.821972 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.821949 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:07:11.822295 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.822271 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-data" (OuterVolumeSpecName: "istio-data") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:07:11.823606 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.823575 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-istio-token" (OuterVolumeSpecName: "istio-token") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:07:11.823606 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.823595 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/379918e7-1f8a-4e70-8877-f095845b39d3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:07:11.823849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.823833 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-kube-api-access-xqwvg" (OuterVolumeSpecName: "kube-api-access-xqwvg") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "kube-api-access-xqwvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:07:11.823958 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.823945 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "379918e7-1f8a-4e70-8877-f095845b39d3" (UID: "379918e7-1f8a-4e70-8877-f095845b39d3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:07:11.922316 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.922284 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/379918e7-1f8a-4e70-8877-f095845b39d3-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.922316 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.922310 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.922316 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.922319 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.922535 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.922330 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqwvg\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-kube-api-access-xqwvg\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.922535 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.922340 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/379918e7-1f8a-4e70-8877-f095845b39d3-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:11.922535 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:11.922347 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/379918e7-1f8a-4e70-8877-f095845b39d3-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:07:12.264159 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.264124 2561 generic.go:356] "Generic (PLEG): container finished" podID="379918e7-1f8a-4e70-8877-f095845b39d3" containerID="fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8" exitCode=0 Jul 28 00:07:12.264323 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.264185 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" Jul 28 00:07:12.264323 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.264202 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" event={"ID":"379918e7-1f8a-4e70-8877-f095845b39d3","Type":"ContainerDied","Data":"fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8"} Jul 28 00:07:12.264323 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.264284 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226" event={"ID":"379918e7-1f8a-4e70-8877-f095845b39d3","Type":"ContainerDied","Data":"f16f4fea4d30905f281f6ee44cbe9d2e45cca39a2788e5849167fd9883735d63"} Jul 28 00:07:12.264323 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.264308 2561 scope.go:117] "RemoveContainer" containerID="fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8" Jul 28 00:07:12.273625 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.273593 2561 scope.go:117] "RemoveContainer" containerID="3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca" Jul 28 00:07:12.281805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.281761 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226"] Jul 28 00:07:12.282958 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.282936 2561 scope.go:117] "RemoveContainer" containerID="fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8" Jul 28 00:07:12.283283 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:07:12.283259 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8\": container with ID starting with fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8 not found: ID does not exist" containerID="fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8" Jul 28 00:07:12.283344 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.283293 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8"} err="failed to get container status \"fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8\": rpc error: code = NotFound desc = could not find container \"fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8\": container with ID starting with fc0eba50d9e03b173e87331aefe7b93b719f61a166f81378bc11c929d59e06b8 not found: ID does not exist" Jul 28 00:07:12.283344 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.283316 2561 scope.go:117] "RemoveContainer" containerID="3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca" Jul 28 00:07:12.283596 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:07:12.283578 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca\": container with ID starting with 3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca not found: ID does not exist" containerID="3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca" Jul 28 00:07:12.283634 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.283602 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca"} err="failed to get container status \"3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca\": rpc error: code = NotFound desc = could not find container \"3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca\": container with ID starting with 3202fdae01f9e7f246cff260851750b5941a63379ffea4a4a6e4e8cc1f0681ca not found: ID does not exist" Jul 28 00:07:12.284935 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:12.284917 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-5b6cfa42-openshift-default-5b559dcb8-lg226"] Jul 28 00:07:13.971845 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:07:13.971810 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" path="/var/lib/kubelet/pods/379918e7-1f8a-4e70-8877-f095845b39d3/volumes" Jul 28 00:08:48.203969 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.203935 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz"] Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204435 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204454 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="c60b42ad-e666-4df1-99dd-af5b2de8608d" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204465 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204471 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204495 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204504 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204524 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204530 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204541 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204547 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204618 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204626 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" Jul 28 00:08:48.204651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.204632 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="975f63bc-6b16-4c8c-bbd4-37396eaec1b9" containerName="istio-proxy" Jul 28 00:08:48.207791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.207768 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.211165 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.211112 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8c8f285e-openshift-default-dockercfg-8wtts\"" Jul 28 00:08:48.236601 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.236575 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz"] Jul 28 00:08:48.278758 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.278727 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-podinfo\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.278941 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.278780 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-certs\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.278941 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.278855 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-token\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.278941 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.278917 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-envoy\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.279130 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.278967 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istiod-ca-cert\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.279130 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.278983 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk7vh\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-kube-api-access-pk7vh\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.279130 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.279016 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-credential-socket\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.279130 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.279047 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-socket\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.279282 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.279140 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-data\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.380613 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380561 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-credential-socket\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.380895 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380621 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-socket\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.380895 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380680 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-data\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.380895 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380710 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-podinfo\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.380895 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380759 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-certs\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.380895 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380813 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-token\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.380895 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380857 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-envoy\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.381298 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380914 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istiod-ca-cert\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.381298 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.380941 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk7vh\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-kube-api-access-pk7vh\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.381298 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.381119 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-credential-socket\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.381298 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.381147 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-socket\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.381612 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.381564 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-certs\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.381748 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.381631 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-data\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.381899 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.381877 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istiod-ca-cert\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.383953 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.383920 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-envoy\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.384191 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.384169 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-podinfo\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.390851 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.390827 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk7vh\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-kube-api-access-pk7vh\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.391633 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.391615 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-token\") pod \"e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.520206 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.520172 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:48.681024 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.680996 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz"] Jul 28 00:08:48.683022 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:08:48.682994 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0ad2ef4_7099_41dc_ae7a_ae1ccf0955fe.slice/crio-46b02a00468feac7f7de85a0d271df91b2096086fbb3e9c1f01e4f09c496d17a WatchSource:0}: Error finding container 46b02a00468feac7f7de85a0d271df91b2096086fbb3e9c1f01e4f09c496d17a: Status 404 returned error can't find the container with id 46b02a00468feac7f7de85a0d271df91b2096086fbb3e9c1f01e4f09c496d17a Jul 28 00:08:48.685640 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.685608 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:08:48.685731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.685671 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:08:48.685731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.685703 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:08:48.764840 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.764811 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" event={"ID":"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe","Type":"ContainerStarted","Data":"a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15"} Jul 28 00:08:48.765071 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.765025 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" event={"ID":"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe","Type":"ContainerStarted","Data":"46b02a00468feac7f7de85a0d271df91b2096086fbb3e9c1f01e4f09c496d17a"} Jul 28 00:08:48.803907 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:48.803805 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" podStartSLOduration=0.803789598 podStartE2EDuration="803.789598ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:08:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:08:48.800573089 +0000 UTC m=+1929.264955228" watchObservedRunningTime="2026-07-28 00:08:48.803789598 +0000 UTC m=+1929.268171737" Jul 28 00:08:49.521287 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:49.521257 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:49.525886 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:49.525861 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:49.771195 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:49.771164 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:08:49.772207 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:08:49.772148 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:09:01.836192 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:01.836161 2561 generic.go:356] "Generic (PLEG): container finished" podID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerID="a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15" exitCode=0 Jul 28 00:09:01.836602 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:01.836238 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" event={"ID":"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe","Type":"ContainerDied","Data":"a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15"} Jul 28 00:09:01.836650 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:01.836615 2561 scope.go:117] "RemoveContainer" containerID="a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15" Jul 28 00:09:01.837758 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:01.837725 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:09:01.837891 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:01.837816 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:09:01.837891 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:01.837862 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:09:02.854276 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:02.854227 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" event={"ID":"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe","Type":"ContainerStarted","Data":"951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce"} Jul 28 00:09:03.521160 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:03.521117 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:09:03.522281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:03.522256 2561 patch_prober.go:28] interesting pod/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 28 00:09:03.522339 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:03.522309 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 28 00:09:04.520583 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:04.520552 2561 patch_prober.go:28] interesting pod/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 28 00:09:04.520973 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:04.520606 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 28 00:09:04.771143 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:04.771034 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:09:05.521320 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:05.521283 2561 patch_prober.go:28] interesting pod/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 28 00:09:05.521708 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:05.521351 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 28 00:09:06.521108 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:06.521048 2561 patch_prober.go:28] interesting pod/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 28 00:09:06.521289 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:06.521125 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 28 00:09:07.523996 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:07.523965 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:09:07.525050 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:07.525029 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:09:24.324020 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:24.323987 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz"] Jul 28 00:09:24.324504 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:24.324227 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" containerID="cri-o://951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce" gracePeriod=30 Jul 28 00:09:29.582409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.582385 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:09:29.674382 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674306 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-envoy\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674382 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674341 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-podinfo\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674382 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674374 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-certs\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674671 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674442 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-token\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674671 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674489 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk7vh\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-kube-api-access-pk7vh\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674671 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674527 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-socket\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674671 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674555 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-data\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674887 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674690 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:09:29.674887 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674706 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-credential-socket\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674887 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674789 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istiod-ca-cert\") pod \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\" (UID: \"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe\") " Jul 28 00:09:29.674887 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674817 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:09:29.675130 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.674899 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:09:29.675201 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.675138 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.675201 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.675157 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.675201 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.675172 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.675338 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.675251 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:09:29.675484 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.675455 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-data" (OuterVolumeSpecName: "istio-data") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:09:29.676687 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.676663 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:09:29.676687 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.676669 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-token" (OuterVolumeSpecName: "istio-token") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:09:29.676839 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.676706 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-kube-api-access-pk7vh" (OuterVolumeSpecName: "kube-api-access-pk7vh") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "kube-api-access-pk7vh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:09:29.677033 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.677017 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" (UID: "a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:09:29.776011 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.775968 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk7vh\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-kube-api-access-pk7vh\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.776011 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.776007 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.776011 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.776017 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.776011 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.776025 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.776298 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.776032 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.776298 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.776039 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:09:29.993090 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.993044 2561 generic.go:356] "Generic (PLEG): container finished" podID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerID="951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce" exitCode=0 Jul 28 00:09:29.993216 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.993091 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" event={"ID":"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe","Type":"ContainerDied","Data":"951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce"} Jul 28 00:09:29.993216 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.993111 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" Jul 28 00:09:29.993216 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.993140 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz" event={"ID":"a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe","Type":"ContainerDied","Data":"46b02a00468feac7f7de85a0d271df91b2096086fbb3e9c1f01e4f09c496d17a"} Jul 28 00:09:29.993216 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:29.993164 2561 scope.go:117] "RemoveContainer" containerID="951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce" Jul 28 00:09:30.002416 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:30.002399 2561 scope.go:117] "RemoveContainer" containerID="a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15" Jul 28 00:09:30.011391 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:30.011300 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz"] Jul 28 00:09:30.012349 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:30.012326 2561 scope.go:117] "RemoveContainer" containerID="951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce" Jul 28 00:09:30.012612 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:09:30.012593 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce\": container with ID starting with 951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce not found: ID does not exist" containerID="951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce" Jul 28 00:09:30.012712 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:30.012622 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce"} err="failed to get container status \"951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce\": rpc error: code = NotFound desc = could not find container \"951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce\": container with ID starting with 951db563fa239af2ec4afe820d5db7358ec9fa515423a22cc0b844059809c9ce not found: ID does not exist" Jul 28 00:09:30.012712 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:30.012640 2561 scope.go:117] "RemoveContainer" containerID="a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15" Jul 28 00:09:30.012917 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:09:30.012898 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15\": container with ID starting with a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15 not found: ID does not exist" containerID="a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15" Jul 28 00:09:30.012965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:30.012920 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15"} err="failed to get container status \"a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15\": rpc error: code = NotFound desc = could not find container \"a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15\": container with ID starting with a349b5bdaed9128eec41541422063729704c0bdecaf358076cc223a3af44de15 not found: ID does not exist" Jul 28 00:09:30.013893 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:30.013876 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8c8f285e-openshift-default-5fd4bd8bd4-bh2pz"] Jul 28 00:09:31.972462 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:31.972388 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" path="/var/lib/kubelet/pods/a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe/volumes" Jul 28 00:09:44.213248 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213216 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6"] Jul 28 00:09:44.213654 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213584 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" Jul 28 00:09:44.213654 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213597 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" Jul 28 00:09:44.213654 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213617 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" Jul 28 00:09:44.213654 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213623 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" Jul 28 00:09:44.213854 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213695 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" Jul 28 00:09:44.213854 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213704 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="379918e7-1f8a-4e70-8877-f095845b39d3" containerName="istio-proxy" Jul 28 00:09:44.213854 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.213710 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="a0ad2ef4-7099-41dc-ae7a-ae1ccf0955fe" containerName="istio-proxy" Jul 28 00:09:44.216824 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.216802 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.219285 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.219261 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-352df9d8-openshift-default-dockercfg-rvzh9\"" Jul 28 00:09:44.227409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.227387 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6"] Jul 28 00:09:44.301613 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301580 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-credential-socket\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.301802 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301622 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-envoy\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.301802 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-podinfo\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.301802 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301685 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlctl\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-kube-api-access-jlctl\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.301802 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301717 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-certs\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.301802 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301754 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946f676a-dbde-4095-ad1b-00d77d5aecab-istiod-ca-cert\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.301802 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301787 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-data\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.302145 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301817 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-socket\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.302145 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.301877 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-token\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.402635 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402598 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-credential-socket\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.402821 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402654 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-envoy\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.402821 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402682 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-podinfo\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.402821 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402706 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlctl\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-kube-api-access-jlctl\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.402821 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402743 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-certs\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.402821 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946f676a-dbde-4095-ad1b-00d77d5aecab-istiod-ca-cert\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.402821 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402817 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-data\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.403149 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402848 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-socket\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.403149 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.402880 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-token\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.403149 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.403100 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-credential-socket\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.403402 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.403370 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-data\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.403837 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.403807 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-certs\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.403837 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.403823 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-socket\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.404093 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.404048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946f676a-dbde-4095-ad1b-00d77d5aecab-istiod-ca-cert\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.405571 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.405538 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-envoy\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.405810 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.405789 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-podinfo\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.410494 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.410470 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-token\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.411086 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.411048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlctl\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-kube-api-access-jlctl\") pod \"e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.527312 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.527276 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:44.661651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.661621 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6"] Jul 28 00:09:44.662980 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:09:44.662957 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946f676a_dbde_4095_ad1b_00d77d5aecab.slice/crio-2fba286589f5432d72ea8714e9f2c3ef4e76b3cc6a893f03aa25287282873281 WatchSource:0}: Error finding container 2fba286589f5432d72ea8714e9f2c3ef4e76b3cc6a893f03aa25287282873281: Status 404 returned error can't find the container with id 2fba286589f5432d72ea8714e9f2c3ef4e76b3cc6a893f03aa25287282873281 Jul 28 00:09:44.667987 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.667949 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:09:44.668050 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.668036 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:09:44.668127 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:44.668110 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:09:45.069107 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:45.069041 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" event={"ID":"946f676a-dbde-4095-ad1b-00d77d5aecab","Type":"ContainerStarted","Data":"50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a"} Jul 28 00:09:45.069281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:45.069125 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" event={"ID":"946f676a-dbde-4095-ad1b-00d77d5aecab","Type":"ContainerStarted","Data":"2fba286589f5432d72ea8714e9f2c3ef4e76b3cc6a893f03aa25287282873281"} Jul 28 00:09:45.089185 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:45.089138 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" podStartSLOduration=1.089123734 podStartE2EDuration="1.089123734s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:09:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:09:45.087810537 +0000 UTC m=+1985.552192679" watchObservedRunningTime="2026-07-28 00:09:45.089123734 +0000 UTC m=+1985.553505950" Jul 28 00:09:45.528099 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:45.528049 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:45.532773 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:45.532750 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:46.074837 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:46.074812 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:09:46.076052 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:09:46.076030 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:10:01.631579 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:01.631017 2561 patch_prober.go:28] interesting pod/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": read tcp 10.132.0.2:58266->10.132.0.46:15021: read: connection reset by peer" start-of-body= Jul 28 00:10:01.631579 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:01.631125 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": read tcp 10.132.0.2:58266->10.132.0.46:15021: read: connection reset by peer" Jul 28 00:10:02.165144 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:02.165106 2561 generic.go:356] "Generic (PLEG): container finished" podID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerID="50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a" exitCode=0 Jul 28 00:10:02.165318 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:02.165179 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" event={"ID":"946f676a-dbde-4095-ad1b-00d77d5aecab","Type":"ContainerDied","Data":"50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a"} Jul 28 00:10:02.165614 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:02.165592 2561 scope.go:117] "RemoveContainer" containerID="50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a" Jul 28 00:10:02.166774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:02.166744 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:10:02.166847 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:02.166830 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:10:02.166894 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:02.166881 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:10:03.172117 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:03.172083 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" event={"ID":"946f676a-dbde-4095-ad1b-00d77d5aecab","Type":"ContainerStarted","Data":"2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294"} Jul 28 00:10:03.527949 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:03.527914 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:10:03.528183 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:03.528161 2561 patch_prober.go:28] interesting pod/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 28 00:10:03.528273 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:03.528237 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 28 00:10:04.527810 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:04.527773 2561 patch_prober.go:28] interesting pod/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 28 00:10:04.528287 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:04.527828 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 28 00:10:05.528536 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:05.528502 2561 patch_prober.go:28] interesting pod/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 28 00:10:05.529174 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:05.529133 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 28 00:10:06.527900 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:06.527862 2561 patch_prober.go:28] interesting pod/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body= Jul 28 00:10:06.528114 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:06.527918 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" Jul 28 00:10:07.531272 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:07.531242 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:10:07.531752 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:07.531504 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:10:07.532150 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:10:07.532134 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:11:40.059165 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:40.059093 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:11:40.070767 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:40.070737 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:11:43.214175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:43.214140 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6"] Jul 28 00:11:43.214666 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:43.214391 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" containerID="cri-o://2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294" gracePeriod=30 Jul 28 00:11:48.469752 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.469725 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:11:48.573243 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573212 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-podinfo\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573243 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573243 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-token\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573494 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573285 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-socket\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573494 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573320 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-certs\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573494 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573338 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-envoy\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573494 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573375 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-data\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573494 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573402 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-credential-socket\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573494 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573436 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946f676a-dbde-4095-ad1b-00d77d5aecab-istiod-ca-cert\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573527 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlctl\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-kube-api-access-jlctl\") pod \"946f676a-dbde-4095-ad1b-00d77d5aecab\" (UID: \"946f676a-dbde-4095-ad1b-00d77d5aecab\") " Jul 28 00:11:48.573816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573667 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:11:48.573816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573780 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:11:48.574025 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573893 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.574025 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.573913 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.574162 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.574093 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:11:48.574162 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.574145 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/946f676a-dbde-4095-ad1b-00d77d5aecab-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:11:48.574422 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.574386 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-data" (OuterVolumeSpecName: "istio-data") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:11:48.575628 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.575599 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-token" (OuterVolumeSpecName: "istio-token") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:11:48.575721 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.575626 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:11:48.575721 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.575647 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:11:48.575721 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.575705 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-kube-api-access-jlctl" (OuterVolumeSpecName: "kube-api-access-jlctl") pod "946f676a-dbde-4095-ad1b-00d77d5aecab" (UID: "946f676a-dbde-4095-ad1b-00d77d5aecab"). InnerVolumeSpecName "kube-api-access-jlctl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:11:48.675292 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.675259 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.675292 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.675286 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.675292 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.675294 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.675292 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.675303 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.675539 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.675311 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/946f676a-dbde-4095-ad1b-00d77d5aecab-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.675539 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.675318 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/946f676a-dbde-4095-ad1b-00d77d5aecab-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.675539 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.675326 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlctl\" (UniqueName: \"kubernetes.io/projected/946f676a-dbde-4095-ad1b-00d77d5aecab-kube-api-access-jlctl\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:11:48.722861 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.722826 2561 generic.go:356] "Generic (PLEG): container finished" podID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerID="2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294" exitCode=0 Jul 28 00:11:48.723037 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.722886 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" Jul 28 00:11:48.723037 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.722904 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" event={"ID":"946f676a-dbde-4095-ad1b-00d77d5aecab","Type":"ContainerDied","Data":"2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294"} Jul 28 00:11:48.723037 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.722967 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6" event={"ID":"946f676a-dbde-4095-ad1b-00d77d5aecab","Type":"ContainerDied","Data":"2fba286589f5432d72ea8714e9f2c3ef4e76b3cc6a893f03aa25287282873281"} Jul 28 00:11:48.723037 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.723003 2561 scope.go:117] "RemoveContainer" containerID="2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294" Jul 28 00:11:48.732766 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.732746 2561 scope.go:117] "RemoveContainer" containerID="50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a" Jul 28 00:11:48.740881 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.740865 2561 scope.go:117] "RemoveContainer" containerID="2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294" Jul 28 00:11:48.741223 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:11:48.741204 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294\": container with ID starting with 2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294 not found: ID does not exist" containerID="2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294" Jul 28 00:11:48.741283 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.741231 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294"} err="failed to get container status \"2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294\": rpc error: code = NotFound desc = could not find container \"2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294\": container with ID starting with 2e26885a17352d19e04424194b53843c0fa54e5b69cd87d71fc071127baae294 not found: ID does not exist" Jul 28 00:11:48.741283 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.741247 2561 scope.go:117] "RemoveContainer" containerID="50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a" Jul 28 00:11:48.741502 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:11:48.741477 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a\": container with ID starting with 50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a not found: ID does not exist" containerID="50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a" Jul 28 00:11:48.741591 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.741505 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a"} err="failed to get container status \"50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a\": rpc error: code = NotFound desc = could not find container \"50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a\": container with ID starting with 50e98c3ae556d1558526ef222e1b4d72ca4fe194e99e647c713292d659ef2d7a not found: ID does not exist" Jul 28 00:11:48.747226 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.747207 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6"] Jul 28 00:11:48.751022 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:48.750997 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-352df9d8-openshift-default-566dcf55d8-8fvl6"] Jul 28 00:11:49.972896 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:11:49.972866 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" path="/var/lib/kubelet/pods/946f676a-dbde-4095-ad1b-00d77d5aecab/volumes" Jul 28 00:12:39.536734 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.536648 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx"] Jul 28 00:12:39.537237 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.537046 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" Jul 28 00:12:39.537237 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.537077 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" Jul 28 00:12:39.537237 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.537088 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" Jul 28 00:12:39.537237 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.537095 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" Jul 28 00:12:39.537237 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.537173 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" Jul 28 00:12:39.540220 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.540204 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.542705 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.542688 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f7352d-openshift-default-dockercfg-j8lvn\"" Jul 28 00:12:39.551419 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.551395 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx"] Jul 28 00:12:39.624550 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624503 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istiod-ca-cert\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.624550 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624555 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-token\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.624808 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624629 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-credential-socket\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.624808 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624680 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-certs\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.624923 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624798 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-socket\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.624923 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624864 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-data\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.624923 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624904 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-podinfo\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.625103 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.624970 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-envoy\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.625103 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.625002 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62wlw\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-kube-api-access-62wlw\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725602 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725571 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-socket\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725619 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-data\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725646 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-podinfo\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725674 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-envoy\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725698 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62wlw\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-kube-api-access-62wlw\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725719 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istiod-ca-cert\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725734 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-token\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.725774 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725765 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-credential-socket\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.726159 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.725813 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-certs\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.726159 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.726049 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-socket\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.726159 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.726109 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-data\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.726330 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.726186 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-certs\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.726423 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.726401 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-credential-socket\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.726570 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.726548 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istiod-ca-cert\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.727966 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.727948 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-envoy\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.728175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.728159 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-podinfo\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.733335 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.733313 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-token\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.733432 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.733395 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62wlw\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-kube-api-access-62wlw\") pod \"e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:39.852808 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.852736 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-f7352d-openshift-default-dockercfg-j8lvn\"" Jul 28 00:12:39.861601 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:39.861581 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:40.002757 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:40.002734 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx"] Jul 28 00:12:40.004857 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:12:40.004829 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c347a1_e547_40dd_b295_4aa4a8b092fb.slice/crio-23e275d38489a5d584945df696731880986a19ff915d94bb0e694125ce0b898d WatchSource:0}: Error finding container 23e275d38489a5d584945df696731880986a19ff915d94bb0e694125ce0b898d: Status 404 returned error can't find the container with id 23e275d38489a5d584945df696731880986a19ff915d94bb0e694125ce0b898d Jul 28 00:12:40.006817 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:40.006784 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:40.006906 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:40.006850 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:40.006906 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:40.006884 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:40.998117 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:40.998076 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" event={"ID":"f6c347a1-e547-40dd-b295-4aa4a8b092fb","Type":"ContainerStarted","Data":"7e1e18668fc192207ece50ee24aab6c985c72b778ba15c9efd47bda1a7d116b9"} Jul 28 00:12:40.998587 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:40.998131 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" event={"ID":"f6c347a1-e547-40dd-b295-4aa4a8b092fb","Type":"ContainerStarted","Data":"23e275d38489a5d584945df696731880986a19ff915d94bb0e694125ce0b898d"} Jul 28 00:12:41.017764 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:41.017715 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" podStartSLOduration=2.017699097 podStartE2EDuration="2.017699097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:12:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:12:41.017396635 +0000 UTC m=+2161.481778775" watchObservedRunningTime="2026-07-28 00:12:41.017699097 +0000 UTC m=+2161.482081236" Jul 28 00:12:41.862149 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:41.862112 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:41.866959 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:41.866929 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:42.003111 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:42.003082 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:42.004094 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:42.004052 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:48.277367 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.277332 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj"] Jul 28 00:12:48.277793 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.277777 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="946f676a-dbde-4095-ad1b-00d77d5aecab" containerName="istio-proxy" Jul 28 00:12:48.280861 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.280844 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.283232 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.283211 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-2ccf60-openshift-default-dockercfg-5qcdd\"" Jul 28 00:12:48.292724 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.292700 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj"] Jul 28 00:12:48.414849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.414809 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istiod-ca-cert\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415037 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.414865 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-envoy\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415037 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.414907 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-certs\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415037 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.414940 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-data\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415253 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.415084 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-podinfo\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415253 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.415151 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-socket\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415253 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.415199 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z5tv\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-kube-api-access-2z5tv\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415359 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.415312 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-credential-socket\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.415359 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.415349 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-token\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516560 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516523 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-podinfo\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516729 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516581 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-socket\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516729 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516624 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z5tv\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-kube-api-access-2z5tv\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516729 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516671 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-credential-socket\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516729 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516711 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-token\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516740 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istiod-ca-cert\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516768 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-envoy\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.516937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516918 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-certs\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.517136 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.516974 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-data\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.517185 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.517143 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-credential-socket\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.517353 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.517326 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-certs\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.517418 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.517352 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-data\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.517418 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.517402 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-socket\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.517524 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.517507 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istiod-ca-cert\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.519434 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.519406 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-envoy\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.519758 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.519737 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-podinfo\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.524461 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.524436 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-token\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.524735 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.524713 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z5tv\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-kube-api-access-2z5tv\") pod \"e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.593444 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.593371 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:48.724878 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.724855 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj"] Jul 28 00:12:48.726787 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:12:48.726758 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1098af60_8ea1_4eb3_8263_2479cc4a0de8.slice/crio-7c86c70ae94ff080474fdb90b5dc6865f05ebab123d896a3e9886344baa3488e WatchSource:0}: Error finding container 7c86c70ae94ff080474fdb90b5dc6865f05ebab123d896a3e9886344baa3488e: Status 404 returned error can't find the container with id 7c86c70ae94ff080474fdb90b5dc6865f05ebab123d896a3e9886344baa3488e Jul 28 00:12:48.729385 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.729350 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:48.729508 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.729439 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:48.729508 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:48.729488 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:49.044729 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:49.044697 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" event={"ID":"1098af60-8ea1-4eb3-8263-2479cc4a0de8","Type":"ContainerStarted","Data":"45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6"} Jul 28 00:12:49.044905 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:49.044747 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" event={"ID":"1098af60-8ea1-4eb3-8263-2479cc4a0de8","Type":"ContainerStarted","Data":"7c86c70ae94ff080474fdb90b5dc6865f05ebab123d896a3e9886344baa3488e"} Jul 28 00:12:49.086275 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:49.086219 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" podStartSLOduration=1.086199227 podStartE2EDuration="1.086199227s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:12:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:12:49.083074954 +0000 UTC m=+2169.547457095" watchObservedRunningTime="2026-07-28 00:12:49.086199227 +0000 UTC m=+2169.550581369" Jul 28 00:12:49.594112 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:49.594081 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:49.598930 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:49.598906 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:50.053990 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:50.053959 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:50.055368 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:50.055346 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:12:56.089941 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:56.089907 2561 generic.go:356] "Generic (PLEG): container finished" podID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerID="7e1e18668fc192207ece50ee24aab6c985c72b778ba15c9efd47bda1a7d116b9" exitCode=0 Jul 28 00:12:56.090367 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:56.089979 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" event={"ID":"f6c347a1-e547-40dd-b295-4aa4a8b092fb","Type":"ContainerDied","Data":"7e1e18668fc192207ece50ee24aab6c985c72b778ba15c9efd47bda1a7d116b9"} Jul 28 00:12:56.090425 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:56.090413 2561 scope.go:117] "RemoveContainer" containerID="7e1e18668fc192207ece50ee24aab6c985c72b778ba15c9efd47bda1a7d116b9" Jul 28 00:12:56.091471 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:56.091445 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:56.091531 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:56.091508 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:56.091575 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:56.091540 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:12:57.004072 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:57.004031 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:57.097135 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:57.097103 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" event={"ID":"f6c347a1-e547-40dd-b295-4aa4a8b092fb","Type":"ContainerStarted","Data":"47228cfa9a35e46ccfd08130fcd7600dadd217dae4bb959ee4c09855c146ffce"} Jul 28 00:12:57.863189 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:57.863152 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:12:57.863557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:57.863528 2561 patch_prober.go:28] interesting pod/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 28 00:12:57.863768 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:57.863742 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 28 00:12:58.862750 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:58.862712 2561 patch_prober.go:28] interesting pod/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 28 00:12:58.863402 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:58.863366 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 28 00:12:59.862797 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:59.862754 2561 patch_prober.go:28] interesting pod/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 28 00:12:59.863446 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:12:59.863411 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 28 00:13:00.862143 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:00.862099 2561 patch_prober.go:28] interesting pod/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 28 00:13:00.862374 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:00.862166 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 28 00:13:01.865954 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:01.865927 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:13:01.866331 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:01.866299 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:13:01.866941 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:01.866920 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:13:05.055466 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.055428 2561 patch_prober.go:28] interesting pod/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 28 00:13:05.055898 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.055487 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 28 00:13:05.141289 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.141256 2561 generic.go:356] "Generic (PLEG): container finished" podID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerID="45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6" exitCode=0 Jul 28 00:13:05.141450 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.141304 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" event={"ID":"1098af60-8ea1-4eb3-8263-2479cc4a0de8","Type":"ContainerDied","Data":"45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6"} Jul 28 00:13:05.141754 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.141740 2561 scope.go:117] "RemoveContainer" containerID="45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6" Jul 28 00:13:05.142754 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.142730 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:13:05.142811 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.142791 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:13:05.142852 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:05.142826 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:13:06.155840 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:06.155803 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" event={"ID":"1098af60-8ea1-4eb3-8263-2479cc4a0de8","Type":"ContainerStarted","Data":"005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504"} Jul 28 00:13:06.593576 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:06.593539 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:13:06.593917 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:06.593894 2561 patch_prober.go:28] interesting pod/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 28 00:13:06.593988 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:06.593941 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 28 00:13:07.593773 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:07.593739 2561 patch_prober.go:28] interesting pod/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 28 00:13:07.594195 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:07.593797 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 28 00:13:08.594549 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:08.594513 2561 patch_prober.go:28] interesting pod/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 28 00:13:08.594957 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:08.594570 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 28 00:13:09.593733 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:09.593695 2561 patch_prober.go:28] interesting pod/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Jul 28 00:13:09.593942 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:09.593750 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Jul 28 00:13:10.598307 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:10.598274 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:13:10.598783 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:10.598523 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:13:10.599251 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:13:10.599233 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:14:16.880560 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:16.880482 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5"] Jul 28 00:14:16.883934 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:16.883918 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" Jul 28 00:14:16.886395 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:16.886372 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 28 00:14:16.886527 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:16.886406 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r9jgn\"" Jul 28 00:14:16.891693 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:16.891669 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5"] Jul 28 00:14:17.068517 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:17.068484 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fae89ee-ec2b-4c02-8810-609432ce3fe1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5\" (UID: \"2fae89ee-ec2b-4c02-8810-609432ce3fe1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" Jul 28 00:14:17.169397 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:17.169311 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fae89ee-ec2b-4c02-8810-609432ce3fe1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5\" (UID: \"2fae89ee-ec2b-4c02-8810-609432ce3fe1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" Jul 28 00:14:17.169915 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:17.169895 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fae89ee-ec2b-4c02-8810-609432ce3fe1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5\" (UID: \"2fae89ee-ec2b-4c02-8810-609432ce3fe1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" Jul 28 00:14:17.196292 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:17.196265 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" Jul 28 00:14:17.333508 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:17.333482 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5"] Jul 28 00:14:17.335596 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:14:17.335568 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fae89ee_ec2b_4c02_8810_609432ce3fe1.slice/crio-8424026c7e513ba7984abd784b90c46f018ac9e3fa297eb01ad88c19adad171f WatchSource:0}: Error finding container 8424026c7e513ba7984abd784b90c46f018ac9e3fa297eb01ad88c19adad171f: Status 404 returned error can't find the container with id 8424026c7e513ba7984abd784b90c46f018ac9e3fa297eb01ad88c19adad171f Jul 28 00:14:17.546095 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:17.546046 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" event={"ID":"2fae89ee-ec2b-4c02-8810-609432ce3fe1","Type":"ContainerStarted","Data":"8424026c7e513ba7984abd784b90c46f018ac9e3fa297eb01ad88c19adad171f"} Jul 28 00:14:18.553291 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:18.553261 2561 generic.go:356] "Generic (PLEG): container finished" podID="2fae89ee-ec2b-4c02-8810-609432ce3fe1" containerID="32d12fdb74f5ac93a0aab695e92b1d8f0148bcd75d3e1d9d0f7e8888df0e76b7" exitCode=0 Jul 28 00:14:18.553698 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:18.553348 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" event={"ID":"2fae89ee-ec2b-4c02-8810-609432ce3fe1","Type":"ContainerDied","Data":"32d12fdb74f5ac93a0aab695e92b1d8f0148bcd75d3e1d9d0f7e8888df0e76b7"} Jul 28 00:14:19.692670 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:19.692646 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" Jul 28 00:14:19.790320 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:19.790288 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fae89ee-ec2b-4c02-8810-609432ce3fe1-maas-api-service-ca\") pod \"2fae89ee-ec2b-4c02-8810-609432ce3fe1\" (UID: \"2fae89ee-ec2b-4c02-8810-609432ce3fe1\") " Jul 28 00:14:19.790617 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:19.790596 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2fae89ee-ec2b-4c02-8810-609432ce3fe1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "2fae89ee-ec2b-4c02-8810-609432ce3fe1" (UID: "2fae89ee-ec2b-4c02-8810-609432ce3fe1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:14:19.891031 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:19.890951 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/2fae89ee-ec2b-4c02-8810-609432ce3fe1-maas-api-service-ca\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:20.564210 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:20.564180 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" Jul 28 00:14:20.564210 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:20.564204 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5" event={"ID":"2fae89ee-ec2b-4c02-8810-609432ce3fe1","Type":"ContainerDied","Data":"8424026c7e513ba7984abd784b90c46f018ac9e3fa297eb01ad88c19adad171f"} Jul 28 00:14:20.564425 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:20.564247 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8424026c7e513ba7984abd784b90c46f018ac9e3fa297eb01ad88c19adad171f" Jul 28 00:14:31.962357 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:31.962321 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5"] Jul 28 00:14:31.964784 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:31.964758 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-f7352d-xcgj5"] Jul 28 00:14:31.972169 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:31.972140 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fae89ee-ec2b-4c02-8810-609432ce3fe1" path="/var/lib/kubelet/pods/2fae89ee-ec2b-4c02-8810-609432ce3fe1/volumes" Jul 28 00:14:38.481530 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:38.481499 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx"] Jul 28 00:14:38.481946 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:38.481734 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" containerID="cri-o://47228cfa9a35e46ccfd08130fcd7600dadd217dae4bb959ee4c09855c146ffce" gracePeriod=30 Jul 28 00:14:43.689444 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.689415 2561 generic.go:356] "Generic (PLEG): container finished" podID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerID="47228cfa9a35e46ccfd08130fcd7600dadd217dae4bb959ee4c09855c146ffce" exitCode=0 Jul 28 00:14:43.689820 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.689453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" event={"ID":"f6c347a1-e547-40dd-b295-4aa4a8b092fb","Type":"ContainerDied","Data":"47228cfa9a35e46ccfd08130fcd7600dadd217dae4bb959ee4c09855c146ffce"} Jul 28 00:14:43.689820 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.689523 2561 scope.go:117] "RemoveContainer" containerID="7e1e18668fc192207ece50ee24aab6c985c72b778ba15c9efd47bda1a7d116b9" Jul 28 00:14:43.744676 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.744620 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:14:43.805595 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805565 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-token\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.805595 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805603 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62wlw\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-kube-api-access-62wlw\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.805844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805663 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-envoy\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.805844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805688 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-credential-socket\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.805844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805706 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-certs\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.805844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805752 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-socket\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.805844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805798 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-data\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.806162 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805848 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istiod-ca-cert\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.806162 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.805872 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-podinfo\") pod \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\" (UID: \"f6c347a1-e547-40dd-b295-4aa4a8b092fb\") " Jul 28 00:14:43.806162 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.806023 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:14:43.806162 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.806033 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:14:43.806433 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.806291 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.806433 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.806329 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.806433 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.806383 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:14:43.806701 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.806668 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:14:43.806836 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.806811 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-data" (OuterVolumeSpecName: "istio-data") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:14:43.807979 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.807950 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:14:43.807979 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.807957 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-token" (OuterVolumeSpecName: "istio-token") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:14:43.808137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.808005 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-kube-api-access-62wlw" (OuterVolumeSpecName: "kube-api-access-62wlw") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "kube-api-access-62wlw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:14:43.808137 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.808091 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f6c347a1-e547-40dd-b295-4aa4a8b092fb" (UID: "f6c347a1-e547-40dd-b295-4aa4a8b092fb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:14:43.906908 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.906876 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.906908 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.906903 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62wlw\" (UniqueName: \"kubernetes.io/projected/f6c347a1-e547-40dd-b295-4aa4a8b092fb-kube-api-access-62wlw\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.906908 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.906913 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.907165 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.906922 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.907165 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.906930 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.907165 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.906939 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:43.907165 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:43.906947 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6c347a1-e547-40dd-b295-4aa4a8b092fb-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:14:44.696245 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:44.696213 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" Jul 28 00:14:44.696710 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:44.696213 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx" event={"ID":"f6c347a1-e547-40dd-b295-4aa4a8b092fb","Type":"ContainerDied","Data":"23e275d38489a5d584945df696731880986a19ff915d94bb0e694125ce0b898d"} Jul 28 00:14:44.696710 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:44.696365 2561 scope.go:117] "RemoveContainer" containerID="47228cfa9a35e46ccfd08130fcd7600dadd217dae4bb959ee4c09855c146ffce" Jul 28 00:14:44.713172 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:44.713150 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx"] Jul 28 00:14:44.718702 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:44.718681 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-f7352d-openshift-default-67f45cd668-mstlx"] Jul 28 00:14:45.972460 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:14:45.972427 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" path="/var/lib/kubelet/pods/f6c347a1-e547-40dd-b295-4aa4a8b092fb/volumes" Jul 28 00:15:00.272954 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:00.272921 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj"] Jul 28 00:15:00.273471 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:00.273225 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" containerID="cri-o://005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504" gracePeriod=30 Jul 28 00:15:01.177965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.177929 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf"] Jul 28 00:15:01.178331 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178317 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" Jul 28 00:15:01.178386 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178333 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" Jul 28 00:15:01.178386 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178347 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" Jul 28 00:15:01.178386 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178353 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" Jul 28 00:15:01.178386 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178360 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fae89ee-ec2b-4c02-8810-609432ce3fe1" containerName="revoke-keys" Jul 28 00:15:01.178386 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178366 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fae89ee-ec2b-4c02-8810-609432ce3fe1" containerName="revoke-keys" Jul 28 00:15:01.178561 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178421 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" Jul 28 00:15:01.178561 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178429 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6c347a1-e547-40dd-b295-4aa4a8b092fb" containerName="istio-proxy" Jul 28 00:15:01.178561 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.178434 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fae89ee-ec2b-4c02-8810-609432ce3fe1" containerName="revoke-keys" Jul 28 00:15:01.181795 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.181779 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.184198 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.184177 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-cd61a5-openshift-default-dockercfg-29t76\"" Jul 28 00:15:01.193268 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.193248 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf"] Jul 28 00:15:01.271721 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.271687 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-data\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.271903 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.271731 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-socket\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.271903 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.271755 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-certs\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.271903 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.271785 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74167634-b328-4816-a651-1cff70e71fd7-istio-podinfo\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.271903 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.271820 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl9x7\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-kube-api-access-hl9x7\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.271903 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.271865 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74167634-b328-4816-a651-1cff70e71fd7-istiod-ca-cert\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.272185 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.271993 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-envoy\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.272185 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.272028 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-istio-token\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.272185 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.272076 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-credential-socket\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.372700 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372663 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-envoy\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.372700 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372700 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-istio-token\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372718 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-credential-socket\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372746 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-data\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372864 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-socket\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372915 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-certs\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372957 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74167634-b328-4816-a651-1cff70e71fd7-istio-podinfo\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.372981 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl9x7\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-kube-api-access-hl9x7\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.373123 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74167634-b328-4816-a651-1cff70e71fd7-istiod-ca-cert\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.373192 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-data\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373728 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.373297 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-credential-socket\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373728 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.373326 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-socket\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373728 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.373386 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-certs\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.373898 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.373769 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74167634-b328-4816-a651-1cff70e71fd7-istiod-ca-cert\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.375163 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.375143 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-envoy\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.375430 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.375412 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74167634-b328-4816-a651-1cff70e71fd7-istio-podinfo\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.380309 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.380282 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-istio-token\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.380654 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.380638 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl9x7\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-kube-api-access-hl9x7\") pod \"e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.492204 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.492121 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:01.631844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.631816 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf"] Jul 28 00:15:01.632979 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:15:01.632956 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74167634_b328_4816_a651_1cff70e71fd7.slice/crio-bdbbdad4865e2310418fd3f55c4ec25683dead3c1b67383147088c21f4ea7e1c WatchSource:0}: Error finding container bdbbdad4865e2310418fd3f55c4ec25683dead3c1b67383147088c21f4ea7e1c: Status 404 returned error can't find the container with id bdbbdad4865e2310418fd3f55c4ec25683dead3c1b67383147088c21f4ea7e1c Jul 28 00:15:01.635399 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.635361 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:01.635485 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.635446 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:01.635531 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.635486 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:01.791163 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.791121 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" event={"ID":"74167634-b328-4816-a651-1cff70e71fd7","Type":"ContainerStarted","Data":"1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869"} Jul 28 00:15:01.791334 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.791181 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" event={"ID":"74167634-b328-4816-a651-1cff70e71fd7","Type":"ContainerStarted","Data":"bdbbdad4865e2310418fd3f55c4ec25683dead3c1b67383147088c21f4ea7e1c"} Jul 28 00:15:01.812691 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:01.812637 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podStartSLOduration=0.812620706 podStartE2EDuration="812.620706ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:15:01.809651429 +0000 UTC m=+2302.274033591" watchObservedRunningTime="2026-07-28 00:15:01.812620706 +0000 UTC m=+2302.277002912" Jul 28 00:15:02.492775 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:02.492741 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:02.497713 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:02.497685 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:02.797224 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:02.797194 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:02.798211 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:02.798190 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:05.532144 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.532119 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:15:05.618241 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618159 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-podinfo\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618241 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618206 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istiod-ca-cert\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618241 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618236 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-data\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618276 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-envoy\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618312 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-credential-socket\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618364 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-token\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618388 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-certs\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618409 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z5tv\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-kube-api-access-2z5tv\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618435 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-socket\") pod \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\" (UID: \"1098af60-8ea1-4eb3-8263-2479cc4a0de8\") " Jul 28 00:15:05.618773 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618703 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:05.618773 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618722 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:15:05.618874 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618817 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:05.618930 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618884 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.618930 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618901 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:05.619046 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.618905 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.619316 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.619287 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-data" (OuterVolumeSpecName: "istio-data") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:05.620440 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.620418 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:15:05.620730 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.620706 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-kube-api-access-2z5tv" (OuterVolumeSpecName: "kube-api-access-2z5tv") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "kube-api-access-2z5tv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:15:05.620798 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.620779 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-token" (OuterVolumeSpecName: "istio-token") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:15:05.620843 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.620801 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1098af60-8ea1-4eb3-8263-2479cc4a0de8" (UID: "1098af60-8ea1-4eb3-8263-2479cc4a0de8"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:05.719593 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.719563 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.719593 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.719588 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.719593 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.719597 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.719822 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.719605 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.719822 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.719614 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z5tv\" (UniqueName: \"kubernetes.io/projected/1098af60-8ea1-4eb3-8263-2479cc4a0de8-kube-api-access-2z5tv\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.719822 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.719622 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1098af60-8ea1-4eb3-8263-2479cc4a0de8-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.719822 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.719630 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1098af60-8ea1-4eb3-8263-2479cc4a0de8-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:05.814405 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.814369 2561 generic.go:356] "Generic (PLEG): container finished" podID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerID="005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504" exitCode=0 Jul 28 00:15:05.814592 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.814427 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" Jul 28 00:15:05.814592 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.814453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" event={"ID":"1098af60-8ea1-4eb3-8263-2479cc4a0de8","Type":"ContainerDied","Data":"005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504"} Jul 28 00:15:05.814592 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.814508 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj" event={"ID":"1098af60-8ea1-4eb3-8263-2479cc4a0de8","Type":"ContainerDied","Data":"7c86c70ae94ff080474fdb90b5dc6865f05ebab123d896a3e9886344baa3488e"} Jul 28 00:15:05.814592 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.814532 2561 scope.go:117] "RemoveContainer" containerID="005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504" Jul 28 00:15:05.823989 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.823971 2561 scope.go:117] "RemoveContainer" containerID="45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6" Jul 28 00:15:05.832002 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.831983 2561 scope.go:117] "RemoveContainer" containerID="005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504" Jul 28 00:15:05.832318 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:15:05.832281 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504\": container with ID starting with 005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504 not found: ID does not exist" containerID="005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504" Jul 28 00:15:05.832448 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.832321 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504"} err="failed to get container status \"005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504\": rpc error: code = NotFound desc = could not find container \"005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504\": container with ID starting with 005d594fa861f86e7bf0bf6cf4971c83785b61931c1c282cf7cfc5dbe53b2504 not found: ID does not exist" Jul 28 00:15:05.832448 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.832345 2561 scope.go:117] "RemoveContainer" containerID="45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6" Jul 28 00:15:05.832841 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:15:05.832804 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6\": container with ID starting with 45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6 not found: ID does not exist" containerID="45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6" Jul 28 00:15:05.832959 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.832857 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6"} err="failed to get container status \"45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6\": rpc error: code = NotFound desc = could not find container \"45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6\": container with ID starting with 45551343b6c49b5cb424daa94fcbe062c969c5d6275c25870a92520a84d2a9d6 not found: ID does not exist" Jul 28 00:15:05.835918 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.835898 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj"] Jul 28 00:15:05.839527 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.839506 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-2ccf60-openshift-default-5b6f74f864-xvtwj"] Jul 28 00:15:05.972474 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:05.972399 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" path="/var/lib/kubelet/pods/1098af60-8ea1-4eb3-8263-2479cc4a0de8/volumes" Jul 28 00:15:09.135949 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.135910 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd"] Jul 28 00:15:09.136396 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.136286 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" Jul 28 00:15:09.136396 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.136298 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" Jul 28 00:15:09.136396 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.136357 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" Jul 28 00:15:09.141149 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.141123 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.143536 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.143514 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-da47a6-openshift-default-dockercfg-bgw67\"" Jul 28 00:15:09.151562 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.151533 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd"] Jul 28 00:15:09.248721 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.248681 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-certs\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.248924 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.248815 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914de716-3ee0-40d8-a95a-be35faeea990-istio-podinfo\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.248924 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.248858 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-credential-socket\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.248924 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.248897 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914de716-3ee0-40d8-a95a-be35faeea990-istiod-ca-cert\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.249129 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.248924 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-istio-token\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.249129 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.248989 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5phg\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-kube-api-access-m5phg\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.249129 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.249017 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-data\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.249129 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.249069 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-envoy\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.249129 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.249103 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-socket\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.349932 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.349893 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-envoy\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.349932 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.349933 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-socket\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350198 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.349965 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-certs\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350198 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350023 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914de716-3ee0-40d8-a95a-be35faeea990-istio-podinfo\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350198 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350078 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-credential-socket\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350198 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350115 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914de716-3ee0-40d8-a95a-be35faeea990-istiod-ca-cert\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350198 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350138 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-istio-token\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350198 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350191 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5phg\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-kube-api-access-m5phg\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350505 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350215 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-data\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350505 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350441 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-socket\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350505 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350486 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-certs\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350670 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350653 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-data\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350767 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350741 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-credential-socket\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.350926 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.350909 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914de716-3ee0-40d8-a95a-be35faeea990-istiod-ca-cert\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.352413 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.352395 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-envoy\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.352496 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.352474 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914de716-3ee0-40d8-a95a-be35faeea990-istio-podinfo\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.360565 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.360537 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-istio-token\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.361170 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.361149 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5phg\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-kube-api-access-m5phg\") pod \"e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.453410 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.453337 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:09.590925 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.590898 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd"] Jul 28 00:15:09.592426 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:15:09.592399 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914de716_3ee0_40d8_a95a_be35faeea990.slice/crio-98d0cebbad534bdb20c9049030b57aabe900e24e0eb25f5f720bef1197a0e5b1 WatchSource:0}: Error finding container 98d0cebbad534bdb20c9049030b57aabe900e24e0eb25f5f720bef1197a0e5b1: Status 404 returned error can't find the container with id 98d0cebbad534bdb20c9049030b57aabe900e24e0eb25f5f720bef1197a0e5b1 Jul 28 00:15:09.594506 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.594474 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:09.594625 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.594536 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:09.594625 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.594562 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:09.839700 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.839658 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" event={"ID":"914de716-3ee0-40d8-a95a-be35faeea990","Type":"ContainerStarted","Data":"3b8e9567ba779fc96d6cc04d8897dd9b66193ff8eacb7a15c0e08cd2430c9501"} Jul 28 00:15:09.839883 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.839730 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" event={"ID":"914de716-3ee0-40d8-a95a-be35faeea990","Type":"ContainerStarted","Data":"98d0cebbad534bdb20c9049030b57aabe900e24e0eb25f5f720bef1197a0e5b1"} Jul 28 00:15:09.858781 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:09.858722 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podStartSLOduration=0.858707164 podStartE2EDuration="858.707164ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:15:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:15:09.858214097 +0000 UTC m=+2310.322596236" watchObservedRunningTime="2026-07-28 00:15:09.858707164 +0000 UTC m=+2310.323089305" Jul 28 00:15:10.454225 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:10.454188 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:10.459132 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:10.459110 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:10.845121 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:10.845091 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:10.846141 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:10.846120 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:18.839224 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:18.839183 2561 patch_prober.go:28] interesting pod/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 00:15:18.839608 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:18.839253 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 00:15:19.900751 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:19.900654 2561 generic.go:356] "Generic (PLEG): container finished" podID="74167634-b328-4816-a651-1cff70e71fd7" containerID="1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869" exitCode=0 Jul 28 00:15:19.900751 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:19.900704 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" event={"ID":"74167634-b328-4816-a651-1cff70e71fd7","Type":"ContainerDied","Data":"1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869"} Jul 28 00:15:19.901247 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:19.901129 2561 scope.go:117] "RemoveContainer" containerID="1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869" Jul 28 00:15:19.902197 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:19.902167 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:19.902265 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:19.902255 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:19.902303 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:19.902293 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:20.908309 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:20.908271 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" event={"ID":"74167634-b328-4816-a651-1cff70e71fd7","Type":"ContainerStarted","Data":"3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93"} Jul 28 00:15:21.492775 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:21.492734 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:21.492989 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:21.492855 2561 patch_prober.go:28] interesting pod/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 28 00:15:21.492989 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:21.492912 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 28 00:15:22.493440 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:22.493396 2561 patch_prober.go:28] interesting pod/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 28 00:15:22.493943 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:22.493456 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 28 00:15:23.493277 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:23.493235 2561 patch_prober.go:28] interesting pod/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 28 00:15:23.493486 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:23.493303 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 28 00:15:24.492642 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:24.492606 2561 patch_prober.go:28] interesting pod/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" start-of-body= Jul 28 00:15:24.492837 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:24.492667 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.50:15021/healthz/ready\": dial tcp 10.132.0.50:15021: connect: connection refused" Jul 28 00:15:25.497999 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:25.497910 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:25.498466 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:25.498230 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:25.499007 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:25.498987 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:26.887213 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:26.887176 2561 patch_prober.go:28] interesting pod/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 00:15:26.887603 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:26.887254 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 00:15:27.957510 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:27.957417 2561 generic.go:356] "Generic (PLEG): container finished" podID="914de716-3ee0-40d8-a95a-be35faeea990" containerID="3b8e9567ba779fc96d6cc04d8897dd9b66193ff8eacb7a15c0e08cd2430c9501" exitCode=0 Jul 28 00:15:27.957510 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:27.957490 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" event={"ID":"914de716-3ee0-40d8-a95a-be35faeea990","Type":"ContainerDied","Data":"3b8e9567ba779fc96d6cc04d8897dd9b66193ff8eacb7a15c0e08cd2430c9501"} Jul 28 00:15:27.957947 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:27.957865 2561 scope.go:117] "RemoveContainer" containerID="3b8e9567ba779fc96d6cc04d8897dd9b66193ff8eacb7a15c0e08cd2430c9501" Jul 28 00:15:27.958893 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:27.958864 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:27.958957 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:27.958934 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:27.959001 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:27.958965 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:15:28.964879 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:28.964844 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" event={"ID":"914de716-3ee0-40d8-a95a-be35faeea990","Type":"ContainerStarted","Data":"181c2f30060aa41877ad98a413a60e918e42dfa4cce374c1a108217e02cbd393"} Jul 28 00:15:29.454121 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:29.454079 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:29.454344 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:29.454219 2561 patch_prober.go:28] interesting pod/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 28 00:15:29.454344 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:29.454294 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 28 00:15:30.454077 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:30.454021 2561 patch_prober.go:28] interesting pod/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 28 00:15:30.454488 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:30.454099 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 28 00:15:31.454205 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:31.454168 2561 patch_prober.go:28] interesting pod/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 28 00:15:31.454624 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:31.454226 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 28 00:15:32.454390 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:32.454307 2561 patch_prober.go:28] interesting pod/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 28 00:15:32.454390 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:32.454367 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 28 00:15:33.457337 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:33.457308 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:33.457826 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:33.457690 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:33.458381 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:33.458363 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:15:51.264468 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:51.264435 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf"] Jul 28 00:15:51.265404 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:51.265344 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" containerID="cri-o://3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93" gracePeriod=30 Jul 28 00:15:55.500675 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:55.500637 2561 patch_prober.go:28] interesting pod/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 00:15:55.501162 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:55.500690 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 00:15:56.518650 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.518625 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:56.607520 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607434 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74167634-b328-4816-a651-1cff70e71fd7-istio-podinfo\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607520 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607478 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74167634-b328-4816-a651-1cff70e71fd7-istiod-ca-cert\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607520 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607515 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-certs\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607536 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-istio-token\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607559 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-socket\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607598 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-data\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607627 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl9x7\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-kube-api-access-hl9x7\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607664 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-credential-socket\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.607791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607694 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-envoy\") pod \"74167634-b328-4816-a651-1cff70e71fd7\" (UID: \"74167634-b328-4816-a651-1cff70e71fd7\") " Jul 28 00:15:56.608154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607890 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:56.608154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607918 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:56.608154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.607961 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74167634-b328-4816-a651-1cff70e71fd7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:15:56.608154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.608126 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74167634-b328-4816-a651-1cff70e71fd7-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.608154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.608146 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.608389 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.608159 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.608389 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.608162 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:56.608718 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.608692 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-data" (OuterVolumeSpecName: "istio-data") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:56.609787 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.609754 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-istio-token" (OuterVolumeSpecName: "istio-token") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:15:56.610085 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.610040 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-kube-api-access-hl9x7" (OuterVolumeSpecName: "kube-api-access-hl9x7") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "kube-api-access-hl9x7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:15:56.610154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.610127 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:15:56.610154 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.610130 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74167634-b328-4816-a651-1cff70e71fd7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74167634-b328-4816-a651-1cff70e71fd7" (UID: "74167634-b328-4816-a651-1cff70e71fd7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:15:56.709306 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.709276 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74167634-b328-4816-a651-1cff70e71fd7-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.709306 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.709301 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.709306 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.709310 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.709542 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.709319 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl9x7\" (UniqueName: \"kubernetes.io/projected/74167634-b328-4816-a651-1cff70e71fd7-kube-api-access-hl9x7\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.709542 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.709328 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:56.709542 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:56.709337 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74167634-b328-4816-a651-1cff70e71fd7-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:15:57.120649 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.120613 2561 generic.go:356] "Generic (PLEG): container finished" podID="74167634-b328-4816-a651-1cff70e71fd7" containerID="3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93" exitCode=0 Jul 28 00:15:57.120864 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.120665 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" Jul 28 00:15:57.120864 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.120706 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" event={"ID":"74167634-b328-4816-a651-1cff70e71fd7","Type":"ContainerDied","Data":"3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93"} Jul 28 00:15:57.120864 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.120776 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf" event={"ID":"74167634-b328-4816-a651-1cff70e71fd7","Type":"ContainerDied","Data":"bdbbdad4865e2310418fd3f55c4ec25683dead3c1b67383147088c21f4ea7e1c"} Jul 28 00:15:57.120864 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.120809 2561 scope.go:117] "RemoveContainer" containerID="3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93" Jul 28 00:15:57.130672 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.130652 2561 scope.go:117] "RemoveContainer" containerID="1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869" Jul 28 00:15:57.138485 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.138468 2561 scope.go:117] "RemoveContainer" containerID="3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93" Jul 28 00:15:57.138729 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:15:57.138707 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93\": container with ID starting with 3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93 not found: ID does not exist" containerID="3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93" Jul 28 00:15:57.138818 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.138734 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93"} err="failed to get container status \"3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93\": rpc error: code = NotFound desc = could not find container \"3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93\": container with ID starting with 3e42481487a5e842fa73a6a3b7a9a4139837fd29ecdefbba54cd94a64aafcc93 not found: ID does not exist" Jul 28 00:15:57.138818 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.138751 2561 scope.go:117] "RemoveContainer" containerID="1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869" Jul 28 00:15:57.138989 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:15:57.138974 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869\": container with ID starting with 1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869 not found: ID does not exist" containerID="1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869" Jul 28 00:15:57.139030 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.138992 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869"} err="failed to get container status \"1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869\": rpc error: code = NotFound desc = could not find container \"1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869\": container with ID starting with 1e6c041e6959420fc186842008023c39c7e639e2a0e1a7ed7c24ae660adff869 not found: ID does not exist" Jul 28 00:15:57.143449 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.143426 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf"] Jul 28 00:15:57.146853 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.146834 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-cd61a5-openshift-default-75676666dd-4fwkf"] Jul 28 00:15:57.972886 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:15:57.972849 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74167634-b328-4816-a651-1cff70e71fd7" path="/var/lib/kubelet/pods/74167634-b328-4816-a651-1cff70e71fd7/volumes" Jul 28 00:16:18.132312 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:18.132277 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd"] Jul 28 00:16:18.132918 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:18.132555 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" containerID="cri-o://181c2f30060aa41877ad98a413a60e918e42dfa4cce374c1a108217e02cbd393" gracePeriod=30 Jul 28 00:16:18.460443 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:18.460330 2561 patch_prober.go:28] interesting pod/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 28 00:16:18.460443 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:18.460402 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 00:16:19.046448 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046418 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w"] Jul 28 00:16:19.046760 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046749 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" Jul 28 00:16:19.046805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046762 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" Jul 28 00:16:19.046805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046784 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" Jul 28 00:16:19.046805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046790 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" Jul 28 00:16:19.046805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046800 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" Jul 28 00:16:19.046805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046806 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" Jul 28 00:16:19.046961 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046859 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="1098af60-8ea1-4eb3-8263-2479cc4a0de8" containerName="istio-proxy" Jul 28 00:16:19.046961 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046870 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" Jul 28 00:16:19.046961 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.046878 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="74167634-b328-4816-a651-1cff70e71fd7" containerName="istio-proxy" Jul 28 00:16:19.049856 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.049836 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.052200 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.052181 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-328ef1-openshift-default-dockercfg-875qc\"" Jul 28 00:16:19.063256 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.063235 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w"] Jul 28 00:16:19.220203 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220167 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-envoy\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220211 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2161d14d-5b58-4759-a328-0702b7fe7be0-istiod-ca-cert\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220268 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-credential-socket\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220287 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-data\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220311 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-certs\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220338 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-podinfo\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220403 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5qz6\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-kube-api-access-p5qz6\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220472 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-socket\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.220585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.220501 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-token\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321401 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321307 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-socket\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321401 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321350 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-token\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321401 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321382 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-envoy\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321704 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321416 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2161d14d-5b58-4759-a328-0702b7fe7be0-istiod-ca-cert\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321704 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321460 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-credential-socket\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321704 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321486 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-data\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321704 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321515 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-certs\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321704 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321546 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-podinfo\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.321704 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321645 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5qz6\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-kube-api-access-p5qz6\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.322027 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321789 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-socket\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.322027 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.321848 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-credential-socket\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.322027 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.322018 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-data\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.322179 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.322126 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-certs\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.322179 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.322166 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2161d14d-5b58-4759-a328-0702b7fe7be0-istiod-ca-cert\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.323672 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.323649 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-envoy\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.324337 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.324314 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-podinfo\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.328832 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.328801 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-token\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.328932 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.328889 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5qz6\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-kube-api-access-p5qz6\") pod \"e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.359410 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.359385 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:19.491669 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.491641 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w"] Jul 28 00:16:19.493794 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:16:19.493761 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2161d14d_5b58_4759_a328_0702b7fe7be0.slice/crio-a20e035193551d65ad4bbc7605a79e2d8e633485791c97d1e12a05e0ff15fbe5 WatchSource:0}: Error finding container a20e035193551d65ad4bbc7605a79e2d8e633485791c97d1e12a05e0ff15fbe5: Status 404 returned error can't find the container with id a20e035193551d65ad4bbc7605a79e2d8e633485791c97d1e12a05e0ff15fbe5 Jul 28 00:16:19.496022 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.495990 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:19.496111 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.496066 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:19.496111 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:19.496098 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:20.246424 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:20.246386 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" event={"ID":"2161d14d-5b58-4759-a328-0702b7fe7be0","Type":"ContainerStarted","Data":"6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd"} Jul 28 00:16:20.246928 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:20.246445 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" event={"ID":"2161d14d-5b58-4759-a328-0702b7fe7be0","Type":"ContainerStarted","Data":"a20e035193551d65ad4bbc7605a79e2d8e633485791c97d1e12a05e0ff15fbe5"} Jul 28 00:16:20.267168 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:20.267119 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podStartSLOduration=1.267102804 podStartE2EDuration="1.267102804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:16:20.264455565 +0000 UTC m=+2380.728837708" watchObservedRunningTime="2026-07-28 00:16:20.267102804 +0000 UTC m=+2380.731484941" Jul 28 00:16:20.359689 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:20.359661 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:20.364833 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:20.364812 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:21.251547 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:21.251518 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:21.252436 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:21.252416 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:23.264795 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.264766 2561 generic.go:356] "Generic (PLEG): container finished" podID="914de716-3ee0-40d8-a95a-be35faeea990" containerID="181c2f30060aa41877ad98a413a60e918e42dfa4cce374c1a108217e02cbd393" exitCode=0 Jul 28 00:16:23.265184 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.264839 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" event={"ID":"914de716-3ee0-40d8-a95a-be35faeea990","Type":"ContainerDied","Data":"181c2f30060aa41877ad98a413a60e918e42dfa4cce374c1a108217e02cbd393"} Jul 28 00:16:23.265184 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.264920 2561 scope.go:117] "RemoveContainer" containerID="3b8e9567ba779fc96d6cc04d8897dd9b66193ff8eacb7a15c0e08cd2430c9501" Jul 28 00:16:23.399355 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.399298 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:16:23.565027 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.564990 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914de716-3ee0-40d8-a95a-be35faeea990-istiod-ca-cert\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565286 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565047 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-envoy\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565286 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565097 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5phg\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-kube-api-access-m5phg\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565286 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565206 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-credential-socket\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565286 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565242 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-istio-token\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565286 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565284 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-socket\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565304 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914de716-3ee0-40d8-a95a-be35faeea990-istio-podinfo\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565386 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-certs\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565416 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914de716-3ee0-40d8-a95a-be35faeea990-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:16:23.565557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565432 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-data\") pod \"914de716-3ee0-40d8-a95a-be35faeea990\" (UID: \"914de716-3ee0-40d8-a95a-be35faeea990\") " Jul 28 00:16:23.565557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565476 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:16:23.565816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565584 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:16:23.565816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565599 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:16:23.565816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565785 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.565816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565802 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/914de716-3ee0-40d8-a95a-be35faeea990-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.565816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565814 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.566108 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.565828 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.566316 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.566298 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-data" (OuterVolumeSpecName: "istio-data") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:16:23.567778 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.567752 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-kube-api-access-m5phg" (OuterVolumeSpecName: "kube-api-access-m5phg") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "kube-api-access-m5phg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:16:23.567778 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.567751 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:16:23.567963 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.567877 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/914de716-3ee0-40d8-a95a-be35faeea990-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:16:23.567963 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.567903 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-istio-token" (OuterVolumeSpecName: "istio-token") pod "914de716-3ee0-40d8-a95a-be35faeea990" (UID: "914de716-3ee0-40d8-a95a-be35faeea990"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:16:23.667273 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.667189 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5phg\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-kube-api-access-m5phg\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.667273 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.667219 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/914de716-3ee0-40d8-a95a-be35faeea990-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.667273 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.667229 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/914de716-3ee0-40d8-a95a-be35faeea990-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.667273 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.667237 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:23.667273 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:23.667245 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/914de716-3ee0-40d8-a95a-be35faeea990-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:16:24.272244 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:24.272210 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" event={"ID":"914de716-3ee0-40d8-a95a-be35faeea990","Type":"ContainerDied","Data":"98d0cebbad534bdb20c9049030b57aabe900e24e0eb25f5f720bef1197a0e5b1"} Jul 28 00:16:24.272694 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:24.272246 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd" Jul 28 00:16:24.272694 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:24.272271 2561 scope.go:117] "RemoveContainer" containerID="181c2f30060aa41877ad98a413a60e918e42dfa4cce374c1a108217e02cbd393" Jul 28 00:16:24.291791 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:24.291769 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd"] Jul 28 00:16:24.295548 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:24.295525 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-da47a6-openshift-default-7c8f59f648-q2mxd"] Jul 28 00:16:25.972252 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:25.972220 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="914de716-3ee0-40d8-a95a-be35faeea990" path="/var/lib/kubelet/pods/914de716-3ee0-40d8-a95a-be35faeea990/volumes" Jul 28 00:16:27.362176 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.362143 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68"] Jul 28 00:16:27.362699 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.362576 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" Jul 28 00:16:27.362699 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.362590 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" Jul 28 00:16:27.362699 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.362687 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" Jul 28 00:16:27.367356 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.367334 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.369687 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.369664 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-6e175a-openshift-default-dockercfg-px5gx\"" Jul 28 00:16:27.377557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.377532 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68"] Jul 28 00:16:27.396167 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.396142 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-data\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.396317 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.396172 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-envoy\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.396317 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.396192 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-certs\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.396317 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.396254 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-podinfo\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.396476 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.396346 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7txv\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-kube-api-access-s7txv\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.396790 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.396605 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7031a990-5196-40dd-8c43-b6136eb1fdaf-istiod-ca-cert\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.397109 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.397085 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-socket\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.397379 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.397356 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-token\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.397618 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.397590 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-credential-socket\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.498958 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.498921 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7031a990-5196-40dd-8c43-b6136eb1fdaf-istiod-ca-cert\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499156 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.498978 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-socket\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499156 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499010 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-token\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499156 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499090 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-credential-socket\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499156 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499126 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-data\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499156 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499152 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-envoy\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499482 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499180 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-certs\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499482 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499211 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-podinfo\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499482 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499267 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7txv\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-kube-api-access-s7txv\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499482 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499461 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-socket\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499674 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499521 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-credential-socket\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499674 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499566 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-data\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499787 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499700 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-certs\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.499787 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.499722 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7031a990-5196-40dd-8c43-b6136eb1fdaf-istiod-ca-cert\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.501652 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.501627 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-envoy\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.501808 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.501791 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-podinfo\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.506529 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.506509 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-token\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.506649 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.506628 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7txv\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-kube-api-access-s7txv\") pod \"e2e-shared-b-6e175a-openshift-default-6649944456-98x68\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.680520 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.680425 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:27.845844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.845820 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68"] Jul 28 00:16:27.847478 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:16:27.847450 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7031a990_5196_40dd_8c43_b6136eb1fdaf.slice/crio-a90c9b178944961f3d6610e61b0355640049341b06f47c06059a011554be5289 WatchSource:0}: Error finding container a90c9b178944961f3d6610e61b0355640049341b06f47c06059a011554be5289: Status 404 returned error can't find the container with id a90c9b178944961f3d6610e61b0355640049341b06f47c06059a011554be5289 Jul 28 00:16:27.849753 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.849717 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:27.849848 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.849797 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:27.849915 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:27.849844 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:28.295844 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:28.295810 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" event={"ID":"7031a990-5196-40dd-8c43-b6136eb1fdaf","Type":"ContainerStarted","Data":"1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9"} Jul 28 00:16:28.296042 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:28.295866 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" event={"ID":"7031a990-5196-40dd-8c43-b6136eb1fdaf","Type":"ContainerStarted","Data":"a90c9b178944961f3d6610e61b0355640049341b06f47c06059a011554be5289"} Jul 28 00:16:28.320596 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:28.320550 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podStartSLOduration=1.320536415 podStartE2EDuration="1.320536415s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:16:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:16:28.318083503 +0000 UTC m=+2388.782465642" watchObservedRunningTime="2026-07-28 00:16:28.320536415 +0000 UTC m=+2388.784918553" Jul 28 00:16:28.681530 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:28.681438 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:28.686592 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:28.686564 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:29.301301 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:29.301272 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:29.302144 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:29.302121 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:36.252928 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.252894 2561 patch_prober.go:28] interesting pod/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 28 00:16:36.253370 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.252964 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 28 00:16:36.344622 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.344585 2561 generic.go:356] "Generic (PLEG): container finished" podID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerID="6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd" exitCode=0 Jul 28 00:16:36.344796 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.344662 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" event={"ID":"2161d14d-5b58-4759-a328-0702b7fe7be0","Type":"ContainerDied","Data":"6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd"} Jul 28 00:16:36.345279 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.345258 2561 scope.go:117] "RemoveContainer" containerID="6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd" Jul 28 00:16:36.346561 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.346529 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:36.346686 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.346615 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:36.346686 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:36.346667 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:37.356006 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:37.355958 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" event={"ID":"2161d14d-5b58-4759-a328-0702b7fe7be0","Type":"ContainerStarted","Data":"c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c"} Jul 28 00:16:37.359932 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:37.359906 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:37.360431 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:37.360201 2561 patch_prober.go:28] interesting pod/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 28 00:16:37.360614 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:37.360592 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 28 00:16:38.361537 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:38.361491 2561 patch_prober.go:28] interesting pod/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 28 00:16:38.362086 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:38.361557 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 28 00:16:39.360070 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:39.360007 2561 patch_prober.go:28] interesting pod/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 28 00:16:39.360350 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:39.360097 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 28 00:16:40.108202 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:40.108171 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:16:40.111022 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:40.111003 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:16:40.361265 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:40.360701 2561 patch_prober.go:28] interesting pod/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 28 00:16:40.361265 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:40.360762 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 28 00:16:41.364417 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:41.364391 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:41.364857 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:41.364771 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:41.365534 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:41.365515 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:16:44.302940 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.302908 2561 patch_prober.go:28] interesting pod/e2e-shared-b-6e175a-openshift-default-6649944456-98x68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 28 00:16:44.303371 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.302974 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 28 00:16:44.406668 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.406586 2561 generic.go:356] "Generic (PLEG): container finished" podID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerID="1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9" exitCode=0 Jul 28 00:16:44.406812 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.406666 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" event={"ID":"7031a990-5196-40dd-8c43-b6136eb1fdaf","Type":"ContainerDied","Data":"1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9"} Jul 28 00:16:44.407103 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.407088 2561 scope.go:117] "RemoveContainer" containerID="1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9" Jul 28 00:16:44.408048 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.408021 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:44.408133 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.408113 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:44.408173 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:44.408145 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:16:45.414113 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:45.414078 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" event={"ID":"7031a990-5196-40dd-8c43-b6136eb1fdaf","Type":"ContainerStarted","Data":"ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9"} Jul 28 00:16:45.681628 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:45.681529 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:45.681787 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:45.681732 2561 patch_prober.go:28] interesting pod/e2e-shared-b-6e175a-openshift-default-6649944456-98x68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 28 00:16:45.681837 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:45.681788 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 28 00:16:46.681414 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:46.681371 2561 patch_prober.go:28] interesting pod/e2e-shared-b-6e175a-openshift-default-6649944456-98x68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 28 00:16:46.681847 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:46.681430 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 28 00:16:47.681693 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:47.681655 2561 patch_prober.go:28] interesting pod/e2e-shared-b-6e175a-openshift-default-6649944456-98x68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 28 00:16:47.682105 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:47.681710 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 28 00:16:48.681507 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:48.681472 2561 patch_prober.go:28] interesting pod/e2e-shared-b-6e175a-openshift-default-6649944456-98x68 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" start-of-body= Jul 28 00:16:48.681715 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:48.681531 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.53:15021/healthz/ready\": dial tcp 10.132.0.53:15021: connect: connection refused" Jul 28 00:16:49.685149 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:49.685112 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:49.685533 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:49.685389 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:16:49.686144 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:16:49.686127 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:20:40.213938 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:20:40.213848 2561 scope.go:117] "RemoveContainer" containerID="32d12fdb74f5ac93a0aab695e92b1d8f0148bcd75d3e1d9d0f7e8888df0e76b7" Jul 28 00:21:40.143847 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:21:40.143767 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:21:40.148576 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:21:40.148552 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:22:01.026795 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.026757 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt"] Jul 28 00:22:01.028261 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.028236 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" Jul 28 00:22:01.028423 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.028410 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" Jul 28 00:22:01.028712 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.028700 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="914de716-3ee0-40d8-a95a-be35faeea990" containerName="istio-proxy" Jul 28 00:22:01.033865 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.033839 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.036690 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.036665 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt"] Jul 28 00:22:01.037842 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.037822 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-328ef1\"/\"openshift-service-ca.crt\"" Jul 28 00:22:01.037944 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.037906 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-328ef1\"/\"kube-root-ca.crt\"" Jul 28 00:22:01.038015 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.037838 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-328ef1\"/\"default-dockercfg-7wvks\"" Jul 28 00:22:01.038102 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.037861 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-328ef1\"/\"sub-test-model-328ef1-kserve-self-signed-certs\"" Jul 28 00:22:01.113737 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.113706 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-home\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.113737 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.113735 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-dshm\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.113936 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.113760 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-model-cache\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.113936 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.113778 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tmp-dir\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.113936 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.113905 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvl8l\" (UniqueName: \"kubernetes.io/projected/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-kube-api-access-fvl8l\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.114033 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.113941 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tls-certs\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215435 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215402 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-home\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215435 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215436 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-dshm\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215662 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215458 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-model-cache\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215714 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tmp-dir\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215807 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215791 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-model-cache\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215875 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215810 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvl8l\" (UniqueName: \"kubernetes.io/projected/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-kube-api-access-fvl8l\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215875 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215860 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-home\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.215987 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.215863 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tls-certs\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.216083 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.216040 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tmp-dir\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.217723 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.217701 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-dshm\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.218267 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.218248 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tls-certs\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.223887 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.223863 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvl8l\" (UniqueName: \"kubernetes.io/projected/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-kube-api-access-fvl8l\") pod \"sub-test-model-328ef1-kserve-5985d7cdf-pw2zt\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.344354 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.344275 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:01.480979 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:01.480950 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt"] Jul 28 00:22:01.481832 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:22:01.481808 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode09f6f89_b5a2_4ff4_967a_ffd94405c25c.slice/crio-754fc7e8f7c7580859d7a55c9e1d84e065b352365e65dc18484f921e62721399 WatchSource:0}: Error finding container 754fc7e8f7c7580859d7a55c9e1d84e065b352365e65dc18484f921e62721399: Status 404 returned error can't find the container with id 754fc7e8f7c7580859d7a55c9e1d84e065b352365e65dc18484f921e62721399 Jul 28 00:22:02.113836 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:02.113804 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" event={"ID":"e09f6f89-b5a2-4ff4-967a-ffd94405c25c","Type":"ContainerStarted","Data":"05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1"} Jul 28 00:22:02.113836 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:02.113846 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" event={"ID":"e09f6f89-b5a2-4ff4-967a-ffd94405c25c","Type":"ContainerStarted","Data":"754fc7e8f7c7580859d7a55c9e1d84e065b352365e65dc18484f921e62721399"} Jul 28 00:22:02.129630 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:02.129551 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" podStartSLOduration=1.129532756 podStartE2EDuration="1.129532756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:22:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:22:02.129003093 +0000 UTC m=+2722.593385234" watchObservedRunningTime="2026-07-28 00:22:02.129532756 +0000 UTC m=+2722.593914896" Jul 28 00:22:11.344399 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:11.344367 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:11.344974 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:11.344411 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:11.356907 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:11.356887 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:22:12.177471 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:22:12.177435 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:23:35.027912 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:35.027836 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt"] Jul 28 00:23:50.047657 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.047613 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" podUID="e09f6f89-b5a2-4ff4-967a-ffd94405c25c" containerName="main" containerID="cri-o://05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1" gracePeriod=45 Jul 28 00:23:50.291085 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.291049 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:23:50.327548 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327479 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tls-certs\") pod \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " Jul 28 00:23:50.327669 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327570 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvl8l\" (UniqueName: \"kubernetes.io/projected/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-kube-api-access-fvl8l\") pod \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " Jul 28 00:23:50.327669 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327643 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tmp-dir\") pod \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " Jul 28 00:23:50.327754 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327676 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-model-cache\") pod \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " Jul 28 00:23:50.327754 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327710 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-dshm\") pod \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " Jul 28 00:23:50.327849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327791 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-home\") pod \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\" (UID: \"e09f6f89-b5a2-4ff4-967a-ffd94405c25c\") " Jul 28 00:23:50.327938 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327907 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e09f6f89-b5a2-4ff4-967a-ffd94405c25c" (UID: "e09f6f89-b5a2-4ff4-967a-ffd94405c25c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:23:50.328035 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.327920 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-model-cache" (OuterVolumeSpecName: "model-cache") pod "e09f6f89-b5a2-4ff4-967a-ffd94405c25c" (UID: "e09f6f89-b5a2-4ff4-967a-ffd94405c25c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:23:50.328194 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.328170 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-home" (OuterVolumeSpecName: "home") pod "e09f6f89-b5a2-4ff4-967a-ffd94405c25c" (UID: "e09f6f89-b5a2-4ff4-967a-ffd94405c25c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:23:50.328264 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.328215 2561 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tmp-dir\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:23:50.328264 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.328235 2561 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-model-cache\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:23:50.329762 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.329733 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e09f6f89-b5a2-4ff4-967a-ffd94405c25c" (UID: "e09f6f89-b5a2-4ff4-967a-ffd94405c25c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 28 00:23:50.329865 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.329798 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-dshm" (OuterVolumeSpecName: "dshm") pod "e09f6f89-b5a2-4ff4-967a-ffd94405c25c" (UID: "e09f6f89-b5a2-4ff4-967a-ffd94405c25c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:23:50.329865 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.329832 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-kube-api-access-fvl8l" (OuterVolumeSpecName: "kube-api-access-fvl8l") pod "e09f6f89-b5a2-4ff4-967a-ffd94405c25c" (UID: "e09f6f89-b5a2-4ff4-967a-ffd94405c25c"). InnerVolumeSpecName "kube-api-access-fvl8l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:23:50.428678 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.428640 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvl8l\" (UniqueName: \"kubernetes.io/projected/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-kube-api-access-fvl8l\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:23:50.428678 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.428674 2561 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-dshm\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:23:50.428678 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.428685 2561 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-home\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:23:50.428907 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.428693 2561 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e09f6f89-b5a2-4ff4-967a-ffd94405c25c-tls-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:23:50.690301 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.690209 2561 generic.go:356] "Generic (PLEG): container finished" podID="e09f6f89-b5a2-4ff4-967a-ffd94405c25c" containerID="05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1" exitCode=0 Jul 28 00:23:50.690301 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.690255 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" Jul 28 00:23:50.690495 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.690301 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" event={"ID":"e09f6f89-b5a2-4ff4-967a-ffd94405c25c","Type":"ContainerDied","Data":"05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1"} Jul 28 00:23:50.690495 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.690343 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt" event={"ID":"e09f6f89-b5a2-4ff4-967a-ffd94405c25c","Type":"ContainerDied","Data":"754fc7e8f7c7580859d7a55c9e1d84e065b352365e65dc18484f921e62721399"} Jul 28 00:23:50.690495 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.690363 2561 scope.go:117] "RemoveContainer" containerID="05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1" Jul 28 00:23:50.699879 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.699856 2561 scope.go:117] "RemoveContainer" containerID="05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1" Jul 28 00:23:50.702355 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:23:50.702332 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1\": container with ID starting with 05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1 not found: ID does not exist" containerID="05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1" Jul 28 00:23:50.702444 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.702362 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1"} err="failed to get container status \"05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1\": rpc error: code = NotFound desc = could not find container \"05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1\": container with ID starting with 05d6a742245a8b2ad3d582966aa4366fb47db6639e02f0ae7b84e6188c912ff1 not found: ID does not exist" Jul 28 00:23:50.712485 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.712463 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt"] Jul 28 00:23:50.714893 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:50.714871 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-328ef1/sub-test-model-328ef1-kserve-5985d7cdf-pw2zt"] Jul 28 00:23:51.972604 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:23:51.972569 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e09f6f89-b5a2-4ff4-967a-ffd94405c25c" path="/var/lib/kubelet/pods/e09f6f89-b5a2-4ff4-967a-ffd94405c25c/volumes" Jul 28 00:26:03.034146 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.034068 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm"] Jul 28 00:26:03.034541 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.034426 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09f6f89-b5a2-4ff4-967a-ffd94405c25c" containerName="main" Jul 28 00:26:03.034541 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.034437 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09f6f89-b5a2-4ff4-967a-ffd94405c25c" containerName="main" Jul 28 00:26:03.034541 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.034497 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="e09f6f89-b5a2-4ff4-967a-ffd94405c25c" containerName="main" Jul 28 00:26:03.037444 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.037426 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.039963 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.039944 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-74721a-openshift-default-dockercfg-2ht69\"" Jul 28 00:26:03.048748 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.048723 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm"] Jul 28 00:26:03.164561 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164531 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-certs\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164561 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164569 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-token\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164781 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164610 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-socket\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164781 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164641 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istiod-ca-cert\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164781 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164664 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pmn86\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-kube-api-access-pmn86\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164781 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164686 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-credential-socket\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164781 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164741 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-podinfo\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164944 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164782 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-data\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.164944 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.164808 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-envoy\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.265543 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265502 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-certs\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.265543 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265552 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-token\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.265813 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265589 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-socket\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.265813 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265606 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istiod-ca-cert\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.265813 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265623 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pmn86\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-kube-api-access-pmn86\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.265813 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265782 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-credential-socket\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266034 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265844 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-podinfo\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266034 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265887 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-data\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266034 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265895 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-certs\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266034 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265974 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-socket\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266034 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.265979 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-envoy\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266335 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.266190 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-credential-socket\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266335 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.266267 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-data\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.266411 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.266335 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istiod-ca-cert\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.268166 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.268145 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-envoy\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.268383 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.268357 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-podinfo\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.274393 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.274372 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-token\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.274588 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.274569 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pmn86\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-kube-api-access-pmn86\") pod \"e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.348623 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.348547 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:03.485760 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.485735 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm"] Jul 28 00:26:03.487444 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:26:03.487417 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb3d1cb_bb37_4a8c_af15_5eb5b2888776.slice/crio-72df6a1274f5d88bc1ea6733fc24cf3313ef1d11753449487d0986eee73adf8e WatchSource:0}: Error finding container 72df6a1274f5d88bc1ea6733fc24cf3313ef1d11753449487d0986eee73adf8e: Status 404 returned error can't find the container with id 72df6a1274f5d88bc1ea6733fc24cf3313ef1d11753449487d0986eee73adf8e Jul 28 00:26:03.490291 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.490257 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:03.490371 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.490326 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:03.490371 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:03.490355 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:04.402533 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:04.402497 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" event={"ID":"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776","Type":"ContainerStarted","Data":"a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4"} Jul 28 00:26:04.402921 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:04.402553 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" event={"ID":"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776","Type":"ContainerStarted","Data":"72df6a1274f5d88bc1ea6733fc24cf3313ef1d11753449487d0986eee73adf8e"} Jul 28 00:26:04.422012 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:04.421963 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" podStartSLOduration=1.421945342 podStartE2EDuration="1.421945342s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:26:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:26:04.419891156 +0000 UTC m=+2964.884273307" watchObservedRunningTime="2026-07-28 00:26:04.421945342 +0000 UTC m=+2964.886327480" Jul 28 00:26:05.349048 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:05.349008 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:05.354155 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:05.354130 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:05.408378 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:05.408343 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:05.409372 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:05.409355 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:21.451220 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:21.451181 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 00:26:21.451620 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:21.451237 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 00:26:22.091908 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.091869 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm"] Jul 28 00:26:22.097639 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.097615 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.099935 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.099911 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-056d1a-openshift-default-dockercfg-7jn8n\"" Jul 28 00:26:22.106566 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.106540 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm"] Jul 28 00:26:22.254560 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254519 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7wgh\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-kube-api-access-t7wgh\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.254746 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254600 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e59eab-3771-423f-8a2b-d107c1721034-istiod-ca-cert\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.254746 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254642 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e59eab-3771-423f-8a2b-d107c1721034-istio-podinfo\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.254746 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254673 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-envoy\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.254885 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254794 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-socket\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.254885 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254847 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-data\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.254974 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254944 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-istio-token\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.254974 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254964 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-credential-socket\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.255048 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.254989 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-certs\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356502 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356421 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-envoy\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356502 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356464 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-socket\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356502 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356486 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-data\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356526 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-istio-token\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356550 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-credential-socket\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356582 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-certs\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356651 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7wgh\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-kube-api-access-t7wgh\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356691 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e59eab-3771-423f-8a2b-d107c1721034-istiod-ca-cert\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.356777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356754 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e59eab-3771-423f-8a2b-d107c1721034-istio-podinfo\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.357100 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.356941 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-data\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.357100 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.357004 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-credential-socket\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.357227 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.357114 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-certs\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.357282 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.357237 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-socket\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.357587 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.357563 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e59eab-3771-423f-8a2b-d107c1721034-istiod-ca-cert\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.358960 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.358936 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-envoy\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.359394 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.359371 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e59eab-3771-423f-8a2b-d107c1721034-istio-podinfo\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.364442 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.364424 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-istio-token\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.364620 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.364599 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7wgh\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-kube-api-access-t7wgh\") pod \"e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.410506 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.410476 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:22.549611 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.549586 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm"] Jul 28 00:26:22.550857 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:26:22.550831 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46e59eab_3771_423f_8a2b_d107c1721034.slice/crio-cb26daafff977640def2980ee7ee47f4e10b61782ebdf75e5766d79e1f28b34c WatchSource:0}: Error finding container cb26daafff977640def2980ee7ee47f4e10b61782ebdf75e5766d79e1f28b34c: Status 404 returned error can't find the container with id cb26daafff977640def2980ee7ee47f4e10b61782ebdf75e5766d79e1f28b34c Jul 28 00:26:22.553299 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.553265 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:22.553373 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.553330 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:22.553373 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:22.553362 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:23.510706 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.510671 2561 generic.go:356] "Generic (PLEG): container finished" podID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerID="a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4" exitCode=0 Jul 28 00:26:23.510932 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.510747 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" event={"ID":"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776","Type":"ContainerDied","Data":"a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4"} Jul 28 00:26:23.511296 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.511277 2561 scope.go:117] "RemoveContainer" containerID="a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4" Jul 28 00:26:23.512384 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.512358 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" event={"ID":"46e59eab-3771-423f-8a2b-d107c1721034","Type":"ContainerStarted","Data":"55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e"} Jul 28 00:26:23.512488 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.512408 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" event={"ID":"46e59eab-3771-423f-8a2b-d107c1721034","Type":"ContainerStarted","Data":"cb26daafff977640def2980ee7ee47f4e10b61782ebdf75e5766d79e1f28b34c"} Jul 28 00:26:23.512488 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.512455 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:23.512603 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.512536 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:23.512603 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.512586 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:23.583256 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:23.582960 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" podStartSLOduration=1.582939697 podStartE2EDuration="1.582939697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 00:26:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:26:23.582296975 +0000 UTC m=+2984.046679113" watchObservedRunningTime="2026-07-28 00:26:23.582939697 +0000 UTC m=+2984.047321848" Jul 28 00:26:24.410925 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:24.410887 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:24.415860 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:24.415839 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:24.519571 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:24.519534 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" event={"ID":"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776","Type":"ContainerStarted","Data":"9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e"} Jul 28 00:26:24.519850 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:24.519822 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:24.520736 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:24.520718 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:25.349816 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:25.349782 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:25.350586 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:25.350483 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 28 00:26:25.350757 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:25.350734 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 28 00:26:26.349742 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:26.349693 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 28 00:26:26.349955 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:26.349781 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 28 00:26:27.349039 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:27.348998 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 28 00:26:27.349438 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:27.349405 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 28 00:26:28.373367 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:28.373336 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:28.373930 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:28.373734 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:28.374687 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:28.374661 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:26:40.187516 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:40.187488 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:26:40.194106 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:40.194078 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:26:40.560197 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:40.560166 2561 patch_prober.go:28] interesting pod/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 28 00:26:40.561173 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:40.560236 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 28 00:26:41.618414 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:41.618385 2561 generic.go:356] "Generic (PLEG): container finished" podID="46e59eab-3771-423f-8a2b-d107c1721034" containerID="55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e" exitCode=0 Jul 28 00:26:41.618830 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:41.618453 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" event={"ID":"46e59eab-3771-423f-8a2b-d107c1721034","Type":"ContainerDied","Data":"55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e"} Jul 28 00:26:41.618830 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:41.618828 2561 scope.go:117] "RemoveContainer" containerID="55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e" Jul 28 00:26:41.619984 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:41.619952 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:41.620044 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:41.620026 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:41.620101 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:41.620072 2561 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236224Ki","pods":"250"} Jul 28 00:26:42.626237 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:42.626202 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" event={"ID":"46e59eab-3771-423f-8a2b-d107c1721034","Type":"ContainerStarted","Data":"1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34"} Jul 28 00:26:43.411111 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:43.411070 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:43.411305 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:43.411279 2561 patch_prober.go:28] interesting pod/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 28 00:26:43.411361 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:43.411342 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 28 00:26:44.411644 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:44.411605 2561 patch_prober.go:28] interesting pod/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 28 00:26:44.412095 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:44.411661 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 28 00:26:45.410845 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:45.410809 2561 patch_prober.go:28] interesting pod/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 28 00:26:45.411029 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:45.410865 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 28 00:26:46.410996 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:46.410955 2561 patch_prober.go:28] interesting pod/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 28 00:26:46.411407 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:46.411016 2561 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 28 00:26:47.414550 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:47.414517 2561 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:47.415025 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:47.414933 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:26:47.415693 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:26:47.415676 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:28:37.713772 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.713736 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6"] Jul 28 00:28:37.717404 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.717387 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" Jul 28 00:28:37.719795 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.719774 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 28 00:28:37.720865 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.720842 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-r9jgn\"" Jul 28 00:28:37.724601 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.724580 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6"] Jul 28 00:28:37.783689 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.783654 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7288705f-3a1b-4caf-82b0-d456a8acbc61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6\" (UID: \"7288705f-3a1b-4caf-82b0-d456a8acbc61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" Jul 28 00:28:37.885177 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.885142 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7288705f-3a1b-4caf-82b0-d456a8acbc61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6\" (UID: \"7288705f-3a1b-4caf-82b0-d456a8acbc61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" Jul 28 00:28:37.885807 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:37.885787 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7288705f-3a1b-4caf-82b0-d456a8acbc61-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6\" (UID: \"7288705f-3a1b-4caf-82b0-d456a8acbc61\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" Jul 28 00:28:38.029127 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:38.029095 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" Jul 28 00:28:38.165425 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:38.165398 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6"] Jul 28 00:28:38.166472 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:28:38.166446 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7288705f_3a1b_4caf_82b0_d456a8acbc61.slice/crio-9e94183a41fd13cc5adf2b9cfea2c65b9d86fd915edeb45d014c153b1a1ee23d WatchSource:0}: Error finding container 9e94183a41fd13cc5adf2b9cfea2c65b9d86fd915edeb45d014c153b1a1ee23d: Status 404 returned error can't find the container with id 9e94183a41fd13cc5adf2b9cfea2c65b9d86fd915edeb45d014c153b1a1ee23d Jul 28 00:28:38.284894 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:38.284806 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" event={"ID":"7288705f-3a1b-4caf-82b0-d456a8acbc61","Type":"ContainerStarted","Data":"9e94183a41fd13cc5adf2b9cfea2c65b9d86fd915edeb45d014c153b1a1ee23d"} Jul 28 00:28:39.292064 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:39.292032 2561 generic.go:356] "Generic (PLEG): container finished" podID="7288705f-3a1b-4caf-82b0-d456a8acbc61" containerID="127df25a12812987f2780fef54947d3c6f9cb797cb7274bfc4b21a4c47503f50" exitCode=0 Jul 28 00:28:39.292464 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:39.292124 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" event={"ID":"7288705f-3a1b-4caf-82b0-d456a8acbc61","Type":"ContainerDied","Data":"127df25a12812987f2780fef54947d3c6f9cb797cb7274bfc4b21a4c47503f50"} Jul 28 00:28:40.426542 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:40.426515 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" Jul 28 00:28:40.510199 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:40.510157 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7288705f-3a1b-4caf-82b0-d456a8acbc61-maas-api-service-ca\") pod \"7288705f-3a1b-4caf-82b0-d456a8acbc61\" (UID: \"7288705f-3a1b-4caf-82b0-d456a8acbc61\") " Jul 28 00:28:40.510499 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:40.510478 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7288705f-3a1b-4caf-82b0-d456a8acbc61-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7288705f-3a1b-4caf-82b0-d456a8acbc61" (UID: "7288705f-3a1b-4caf-82b0-d456a8acbc61"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:28:40.611575 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:40.611497 2561 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7288705f-3a1b-4caf-82b0-d456a8acbc61-maas-api-service-ca\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:28:41.304021 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:41.303991 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" Jul 28 00:28:41.304220 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:41.304030 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6" event={"ID":"7288705f-3a1b-4caf-82b0-d456a8acbc61","Type":"ContainerDied","Data":"9e94183a41fd13cc5adf2b9cfea2c65b9d86fd915edeb45d014c153b1a1ee23d"} Jul 28 00:28:41.304220 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:41.304096 2561 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e94183a41fd13cc5adf2b9cfea2c65b9d86fd915edeb45d014c153b1a1ee23d" Jul 28 00:28:57.787454 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:57.787412 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6"] Jul 28 00:28:57.789760 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:57.789736 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-74721a-79wj6"] Jul 28 00:28:57.972175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:28:57.972140 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7288705f-3a1b-4caf-82b0-d456a8acbc61" path="/var/lib/kubelet/pods/7288705f-3a1b-4caf-82b0-d456a8acbc61/volumes" Jul 28 00:30:10.127205 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:10.127171 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm"] Jul 28 00:30:10.127650 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:10.127380 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" containerID="cri-o://9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e" gracePeriod=30 Jul 28 00:30:13.375692 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:13.375654 2561 patch_prober.go:28] interesting pod/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 00:30:13.376087 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:13.375717 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 00:30:15.392694 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.392634 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:30:15.414801 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.414720 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istiod-ca-cert\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.414866 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-certs\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.414896 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-data\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.414930 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pmn86\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-kube-api-access-pmn86\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.414967 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-socket\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.414994 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-credential-socket\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415020 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-envoy\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415153 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-podinfo\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415473 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415189 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-token\") pod \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\" (UID: \"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776\") " Jul 28 00:30:15.415937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415488 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:30:15.415937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415611 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:15.415937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415674 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.415937 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415818 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:15.416174 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.415999 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:15.416787 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.416751 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-data" (OuterVolumeSpecName: "istio-data") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:15.418550 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.417897 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-kube-api-access-pmn86" (OuterVolumeSpecName: "kube-api-access-pmn86") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "kube-api-access-pmn86". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:30:15.418839 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.418814 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-token" (OuterVolumeSpecName: "istio-token") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:30:15.419101 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.419045 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:15.419346 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.419322 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" (UID: "bfb3d1cb-bb37-4a8c-af15-5eb5b2888776"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:30:15.516324 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516296 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.516324 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516323 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.516512 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516334 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pmn86\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-kube-api-access-pmn86\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.516512 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516347 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.516512 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516359 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.516512 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516369 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.516512 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516380 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.516512 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.516391 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:15.829659 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.829626 2561 generic.go:356] "Generic (PLEG): container finished" podID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerID="9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e" exitCode=0 Jul 28 00:30:15.829832 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.829679 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" Jul 28 00:30:15.829832 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.829708 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" event={"ID":"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776","Type":"ContainerDied","Data":"9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e"} Jul 28 00:30:15.829832 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.829756 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm" event={"ID":"bfb3d1cb-bb37-4a8c-af15-5eb5b2888776","Type":"ContainerDied","Data":"72df6a1274f5d88bc1ea6733fc24cf3313ef1d11753449487d0986eee73adf8e"} Jul 28 00:30:15.829832 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.829778 2561 scope.go:117] "RemoveContainer" containerID="9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e" Jul 28 00:30:15.841761 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.841747 2561 scope.go:117] "RemoveContainer" containerID="a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4" Jul 28 00:30:15.850550 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.850531 2561 scope.go:117] "RemoveContainer" containerID="9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e" Jul 28 00:30:15.850813 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:30:15.850793 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e\": container with ID starting with 9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e not found: ID does not exist" containerID="9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e" Jul 28 00:30:15.850884 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.850821 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e"} err="failed to get container status \"9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e\": rpc error: code = NotFound desc = could not find container \"9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e\": container with ID starting with 9ff8306f4a3f21b75c03ac25de8ab242b3bfdadaa72a618e12627d099635995e not found: ID does not exist" Jul 28 00:30:15.850884 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.850838 2561 scope.go:117] "RemoveContainer" containerID="a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4" Jul 28 00:30:15.851140 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:30:15.851117 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4\": container with ID starting with a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4 not found: ID does not exist" containerID="a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4" Jul 28 00:30:15.851229 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.851144 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4"} err="failed to get container status \"a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4\": rpc error: code = NotFound desc = could not find container \"a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4\": container with ID starting with a2d202186e798a4f92fddadded293affda955fdeb372ecd9abe40951ddebdef4 not found: ID does not exist" Jul 28 00:30:15.852849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.852827 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm"] Jul 28 00:30:15.856140 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.856121 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-74721a-openshift-default-65b49b44c4-z2wtm"] Jul 28 00:30:15.972485 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:15.972454 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" path="/var/lib/kubelet/pods/bfb3d1cb-bb37-4a8c-af15-5eb5b2888776/volumes" Jul 28 00:30:32.421281 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:32.421205 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm"] Jul 28 00:30:32.927631 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:32.927571 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" containerID="cri-o://1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34" gracePeriod=30 Jul 28 00:30:38.190956 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.190898 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:30:38.335946 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.335909 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-data\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.335973 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-socket\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.335990 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-istio-token\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336008 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-certs\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336101 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-credential-socket\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336221 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e59eab-3771-423f-8a2b-d107c1721034-istiod-ca-cert\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336292 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:38.336409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336299 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e59eab-3771-423f-8a2b-d107c1721034-istio-podinfo\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336409 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336375 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-envoy\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336625 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336398 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7wgh\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-kube-api-access-t7wgh\") pod \"46e59eab-3771-423f-8a2b-d107c1721034\" (UID: \"46e59eab-3771-423f-8a2b-d107c1721034\") " Jul 28 00:30:38.336625 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336483 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:38.336625 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336592 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46e59eab-3771-423f-8a2b-d107c1721034-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:30:38.336777 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336662 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:38.336848 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336824 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-data" (OuterVolumeSpecName: "istio-data") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:38.336908 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336828 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.336908 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336878 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/46e59eab-3771-423f-8a2b-d107c1721034-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.336908 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336898 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.337095 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.336912 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.338443 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.338413 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:38.338443 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.338413 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/46e59eab-3771-423f-8a2b-d107c1721034-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:30:38.338595 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.338508 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-istio-token" (OuterVolumeSpecName: "istio-token") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:30:38.338595 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.338524 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-kube-api-access-t7wgh" (OuterVolumeSpecName: "kube-api-access-t7wgh") pod "46e59eab-3771-423f-8a2b-d107c1721034" (UID: "46e59eab-3771-423f-8a2b-d107c1721034"). InnerVolumeSpecName "kube-api-access-t7wgh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:30:38.438153 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.438122 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/46e59eab-3771-423f-8a2b-d107c1721034-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.438153 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.438152 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.438354 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.438165 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7wgh\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-kube-api-access-t7wgh\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.438354 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.438178 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/46e59eab-3771-423f-8a2b-d107c1721034-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.438354 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.438190 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/46e59eab-3771-423f-8a2b-d107c1721034-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:38.961105 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.961041 2561 generic.go:356] "Generic (PLEG): container finished" podID="46e59eab-3771-423f-8a2b-d107c1721034" containerID="1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34" exitCode=0 Jul 28 00:30:38.961105 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.961087 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" event={"ID":"46e59eab-3771-423f-8a2b-d107c1721034","Type":"ContainerDied","Data":"1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34"} Jul 28 00:30:38.961105 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.961111 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" Jul 28 00:30:38.961383 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.961145 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm" event={"ID":"46e59eab-3771-423f-8a2b-d107c1721034","Type":"ContainerDied","Data":"cb26daafff977640def2980ee7ee47f4e10b61782ebdf75e5766d79e1f28b34c"} Jul 28 00:30:38.961383 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.961180 2561 scope.go:117] "RemoveContainer" containerID="1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34" Jul 28 00:30:38.971222 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.971204 2561 scope.go:117] "RemoveContainer" containerID="55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e" Jul 28 00:30:38.979365 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.979346 2561 scope.go:117] "RemoveContainer" containerID="1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34" Jul 28 00:30:38.979707 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:30:38.979676 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34\": container with ID starting with 1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34 not found: ID does not exist" containerID="1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34" Jul 28 00:30:38.979860 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.979717 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34"} err="failed to get container status \"1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34\": rpc error: code = NotFound desc = could not find container \"1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34\": container with ID starting with 1d8e07e0295253433e26eadc7cff9f7a98a3fd961a4e4f8ec9cf1b19d1a76b34 not found: ID does not exist" Jul 28 00:30:38.979860 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.979738 2561 scope.go:117] "RemoveContainer" containerID="55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e" Jul 28 00:30:38.980090 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:30:38.980040 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e\": container with ID starting with 55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e not found: ID does not exist" containerID="55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e" Jul 28 00:30:38.980090 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.980082 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e"} err="failed to get container status \"55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e\": rpc error: code = NotFound desc = could not find container \"55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e\": container with ID starting with 55256babc27fdcd64112204e839be27b9204a8def1b0ddc3e503b6a66836be0e not found: ID does not exist" Jul 28 00:30:38.982882 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.982861 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm"] Jul 28 00:30:38.986734 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:38.986712 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-056d1a-openshift-default-5ff6b9dd66-2gdsm"] Jul 28 00:30:39.972512 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:39.972482 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46e59eab-3771-423f-8a2b-d107c1721034" path="/var/lib/kubelet/pods/46e59eab-3771-423f-8a2b-d107c1721034/volumes" Jul 28 00:30:54.231666 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:54.231631 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w"] Jul 28 00:30:54.232121 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:54.231933 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" containerID="cri-o://c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c" gracePeriod=30 Jul 28 00:30:56.367287 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:56.367251 2561 patch_prober.go:28] interesting pod/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 00:30:56.367677 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:56.367308 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 00:30:59.484635 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.484613 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:30:59.535190 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535157 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-socket\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535345 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535212 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5qz6\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-kube-api-access-p5qz6\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535345 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535260 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-envoy\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535345 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535285 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-token\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535480 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535381 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2161d14d-5b58-4759-a328-0702b7fe7be0-istiod-ca-cert\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535480 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535406 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-podinfo\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535480 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535393 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:59.535480 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535443 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-credential-socket\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535480 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535466 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-certs\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535733 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535505 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-data\") pod \"2161d14d-5b58-4759-a328-0702b7fe7be0\" (UID: \"2161d14d-5b58-4759-a328-0702b7fe7be0\") " Jul 28 00:30:59.535733 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535718 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:59.535867 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535836 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:59.535939 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535898 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.535939 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535908 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2161d14d-5b58-4759-a328-0702b7fe7be0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:30:59.535939 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.535922 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.536566 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.536543 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-data" (OuterVolumeSpecName: "istio-data") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:59.537617 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.537596 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-kube-api-access-p5qz6" (OuterVolumeSpecName: "kube-api-access-p5qz6") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "kube-api-access-p5qz6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:30:59.537795 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.537769 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:30:59.537901 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.537877 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:30:59.537901 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.537879 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-token" (OuterVolumeSpecName: "istio-token") pod "2161d14d-5b58-4759-a328-0702b7fe7be0" (UID: "2161d14d-5b58-4759-a328-0702b7fe7be0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:30:59.636533 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.636505 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2161d14d-5b58-4759-a328-0702b7fe7be0-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.636533 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.636528 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.636533 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.636536 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.636747 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.636546 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.636747 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.636554 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5qz6\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-kube-api-access-p5qz6\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.636747 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.636562 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:30:59.636747 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:30:59.636572 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2161d14d-5b58-4759-a328-0702b7fe7be0-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:00.075859 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.075824 2561 generic.go:356] "Generic (PLEG): container finished" podID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerID="c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c" exitCode=0 Jul 28 00:31:00.076014 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.075879 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" Jul 28 00:31:00.076014 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.075911 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" event={"ID":"2161d14d-5b58-4759-a328-0702b7fe7be0","Type":"ContainerDied","Data":"c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c"} Jul 28 00:31:00.076014 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.075968 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w" event={"ID":"2161d14d-5b58-4759-a328-0702b7fe7be0","Type":"ContainerDied","Data":"a20e035193551d65ad4bbc7605a79e2d8e633485791c97d1e12a05e0ff15fbe5"} Jul 28 00:31:00.076014 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.075993 2561 scope.go:117] "RemoveContainer" containerID="c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c" Jul 28 00:31:00.085158 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.085135 2561 scope.go:117] "RemoveContainer" containerID="6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd" Jul 28 00:31:00.092397 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.092377 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w"] Jul 28 00:31:00.093696 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.093682 2561 scope.go:117] "RemoveContainer" containerID="c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c" Jul 28 00:31:00.093956 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:31:00.093937 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c\": container with ID starting with c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c not found: ID does not exist" containerID="c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c" Jul 28 00:31:00.094001 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.093962 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c"} err="failed to get container status \"c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c\": rpc error: code = NotFound desc = could not find container \"c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c\": container with ID starting with c68ad7aa2c69569a46f1b4065260f1641f95a6ee8f4fdd1fe638cc466654038c not found: ID does not exist" Jul 28 00:31:00.094001 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.093979 2561 scope.go:117] "RemoveContainer" containerID="6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd" Jul 28 00:31:00.094231 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:31:00.094213 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd\": container with ID starting with 6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd not found: ID does not exist" containerID="6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd" Jul 28 00:31:00.094277 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.094236 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd"} err="failed to get container status \"6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd\": rpc error: code = NotFound desc = could not find container \"6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd\": container with ID starting with 6f616c3785fd8ab1c13102a0dc0c62afb558f0bc2f8b3761bbc5709e5a6cd1cd not found: ID does not exist" Jul 28 00:31:00.099031 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:00.099010 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-328ef1-openshift-default-c4d67555b-fsh8w"] Jul 28 00:31:01.972638 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:01.972600 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" path="/var/lib/kubelet/pods/2161d14d-5b58-4759-a328-0702b7fe7be0/volumes" Jul 28 00:31:16.169699 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:16.169663 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68"] Jul 28 00:31:16.170246 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:16.169868 2561 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" containerID="cri-o://ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9" gracePeriod=30 Jul 28 00:31:19.687840 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:19.687808 2561 patch_prober.go:28] interesting pod/e2e-shared-b-6e175a-openshift-default-6649944456-98x68 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 28 00:31:19.688241 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:19.687872 2561 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 28 00:31:21.418794 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.418770 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:31:21.445877 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.445816 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-podinfo\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.445877 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.445859 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7031a990-5196-40dd-8c43-b6136eb1fdaf-istiod-ca-cert\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446089 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.445891 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-envoy\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446089 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446070 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7txv\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-kube-api-access-s7txv\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446212 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446111 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-certs\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446212 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446142 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-data\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446212 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446166 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-token\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446372 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446231 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-credential-socket\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446372 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446271 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7031a990-5196-40dd-8c43-b6136eb1fdaf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 28 00:31:21.446372 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446288 2561 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-socket\") pod \"7031a990-5196-40dd-8c43-b6136eb1fdaf\" (UID: \"7031a990-5196-40dd-8c43-b6136eb1fdaf\") " Jul 28 00:31:21.446514 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446463 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:31:21.446805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446646 2561 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.446805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446654 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:31:21.446805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446665 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:31:21.446805 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.446672 2561 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7031a990-5196-40dd-8c43-b6136eb1fdaf-istiod-ca-cert\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.447429 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.447386 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-data" (OuterVolumeSpecName: "istio-data") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:31:21.448407 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.448381 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-token" (OuterVolumeSpecName: "istio-token") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:31:21.448557 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.448534 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 28 00:31:21.448609 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.448534 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 28 00:31:21.448693 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.448671 2561 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-kube-api-access-s7txv" (OuterVolumeSpecName: "kube-api-access-s7txv") pod "7031a990-5196-40dd-8c43-b6136eb1fdaf" (UID: "7031a990-5196-40dd-8c43-b6136eb1fdaf"). InnerVolumeSpecName "kube-api-access-s7txv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 28 00:31:21.547275 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.547245 2561 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-podinfo\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.547275 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.547272 2561 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-envoy\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.547463 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.547282 2561 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7txv\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-kube-api-access-s7txv\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.547463 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.547292 2561 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-workload-certs\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.547463 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.547300 2561 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-data\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.547463 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.547308 2561 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7031a990-5196-40dd-8c43-b6136eb1fdaf-istio-token\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:21.547463 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:21.547316 2561 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7031a990-5196-40dd-8c43-b6136eb1fdaf-credential-socket\") on node \"ip-10-0-142-203.ec2.internal\" DevicePath \"\"" Jul 28 00:31:22.196434 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.196401 2561 generic.go:356] "Generic (PLEG): container finished" podID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerID="ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9" exitCode=0 Jul 28 00:31:22.196602 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.196455 2561 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" Jul 28 00:31:22.196602 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.196476 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" event={"ID":"7031a990-5196-40dd-8c43-b6136eb1fdaf","Type":"ContainerDied","Data":"ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9"} Jul 28 00:31:22.196602 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.196524 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68" event={"ID":"7031a990-5196-40dd-8c43-b6136eb1fdaf","Type":"ContainerDied","Data":"a90c9b178944961f3d6610e61b0355640049341b06f47c06059a011554be5289"} Jul 28 00:31:22.196602 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.196546 2561 scope.go:117] "RemoveContainer" containerID="ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9" Jul 28 00:31:22.205278 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.205262 2561 scope.go:117] "RemoveContainer" containerID="1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9" Jul 28 00:31:22.214411 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.214389 2561 scope.go:117] "RemoveContainer" containerID="ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9" Jul 28 00:31:22.214690 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:31:22.214662 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9\": container with ID starting with ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9 not found: ID does not exist" containerID="ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9" Jul 28 00:31:22.214800 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.214697 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9"} err="failed to get container status \"ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9\": rpc error: code = NotFound desc = could not find container \"ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9\": container with ID starting with ee6a2ececb0ac7dd3719f44fff441c331192a2bac6279538efab17d064998cd9 not found: ID does not exist" Jul 28 00:31:22.214800 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.214733 2561 scope.go:117] "RemoveContainer" containerID="1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9" Jul 28 00:31:22.215386 ip-10-0-142-203 kubenswrapper[2561]: E0728 00:31:22.215349 2561 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9\": container with ID starting with 1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9 not found: ID does not exist" containerID="1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9" Jul 28 00:31:22.215491 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.215382 2561 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9"} err="failed to get container status \"1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9\": rpc error: code = NotFound desc = could not find container \"1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9\": container with ID starting with 1c5206f24f1c4eb2718ed9ff4eaff968cb17f25d137ddfd163a9b22daa4268a9 not found: ID does not exist" Jul 28 00:31:22.215764 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.215746 2561 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68"] Jul 28 00:31:22.218113 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:22.218096 2561 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-6e175a-openshift-default-6649944456-98x68"] Jul 28 00:31:23.171136 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:23.171113 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-458ng_d45f0828-7f4b-48ef-a200-217de6272e42/copy-manifests/0.log" Jul 28 00:31:23.184411 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:23.184389 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-458ng_d45f0828-7f4b-48ef-a200-217de6272e42/manager/0.log" Jul 28 00:31:23.912804 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:23.912772 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29753280-zct62_9658a57c-aa19-4350-922f-df954415fb2f/cleanup/0.log" Jul 28 00:31:23.971734 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:23.971707 2561 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" path="/var/lib/kubelet/pods/7031a990-5196-40dd-8c43-b6136eb1fdaf/volumes" Jul 28 00:31:25.095832 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.095806 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7_745631ec-b4c9-44ea-a2d5-54163c6dd2b8/util/0.log" Jul 28 00:31:25.102659 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.102641 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7_745631ec-b4c9-44ea-a2d5-54163c6dd2b8/pull/0.log" Jul 28 00:31:25.110479 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.110462 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7_745631ec-b4c9-44ea-a2d5-54163c6dd2b8/extract/0.log" Jul 28 00:31:25.226200 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.226173 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk_ca799efc-6596-43e5-8a4d-ea414e26c74c/util/0.log" Jul 28 00:31:25.233066 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.233038 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk_ca799efc-6596-43e5-8a4d-ea414e26c74c/pull/0.log" Jul 28 00:31:25.239936 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.239920 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk_ca799efc-6596-43e5-8a4d-ea414e26c74c/extract/0.log" Jul 28 00:31:25.355178 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.355102 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks_8020da7b-b59a-4df8-a757-52959e50ba4d/util/0.log" Jul 28 00:31:25.362704 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.362682 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks_8020da7b-b59a-4df8-a757-52959e50ba4d/pull/0.log" Jul 28 00:31:25.369293 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.369276 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks_8020da7b-b59a-4df8-a757-52959e50ba4d/extract/0.log" Jul 28 00:31:25.483223 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.483195 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz_8b32f2cd-d210-423c-8266-cafb77073608/pull/0.log" Jul 28 00:31:25.489789 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.489761 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz_8b32f2cd-d210-423c-8266-cafb77073608/extract/0.log" Jul 28 00:31:25.496221 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:25.496200 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz_8b32f2cd-d210-423c-8266-cafb77073608/util/0.log" Jul 28 00:31:26.858211 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:26.858179 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f_4e81ccc9-a64d-441c-bf8f-9c8b97eadc61/istio-proxy/0.log" Jul 28 00:31:26.986842 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:26.986811 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g9fpz_618365d4-2133-4122-a7aa-cc2196ed69f3/discovery/0.log" Jul 28 00:31:27.354923 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:27.354889 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jgqbk_0317dd4a-de8a-4245-887f-310a25b7d581/istio-proxy/1.log" Jul 28 00:31:27.746487 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:27.746416 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c94fdb577-j6wm7_960554c4-9d24-4732-bafa-3d829e451e2a/router/0.log" Jul 28 00:31:28.095938 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:28.095909 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-76mvn_cd9c2b70-de5c-4eea-914e-325a4277d18e/main/0.log" Jul 28 00:31:28.210492 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:28.210462 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-vwjr4_49898aa6-1b9f-4574-8d7e-8d568c59dc72/main/0.log" Jul 28 00:31:28.664962 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:28.664930 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-ktbsz_25f92c31-745a-4bf0-b49b-4961931de18d/main/0.log" Jul 28 00:31:37.560873 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:37.560840 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-25t97_d81e8510-04d0-4c8e-8b56-281f0ccdd165/global-pull-secret-syncer/0.log" Jul 28 00:31:37.772175 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:37.772145 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-xkg6r_6c122ef5-5ba8-4905-8765-e5a6e3c9e2f5/konnectivity-agent/0.log" Jul 28 00:31:37.840382 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:37.840307 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-203.ec2.internal_0c2a396ccbfd4c8c59eadf247f295e9f/haproxy/0.log" Jul 28 00:31:40.221540 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:40.221509 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:31:40.230131 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:40.230106 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:31:41.982523 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:41.982496 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7_745631ec-b4c9-44ea-a2d5-54163c6dd2b8/extract/0.log" Jul 28 00:31:42.008229 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.008206 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7_745631ec-b4c9-44ea-a2d5-54163c6dd2b8/util/0.log" Jul 28 00:31:42.029543 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.029520 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759xdzz7_745631ec-b4c9-44ea-a2d5-54163c6dd2b8/pull/0.log" Jul 28 00:31:42.060424 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.060404 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk_ca799efc-6596-43e5-8a4d-ea414e26c74c/extract/0.log" Jul 28 00:31:42.086202 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.086181 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk_ca799efc-6596-43e5-8a4d-ea414e26c74c/util/0.log" Jul 28 00:31:42.106856 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.106834 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0425jk_ca799efc-6596-43e5-8a4d-ea414e26c74c/pull/0.log" Jul 28 00:31:42.138158 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.138138 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks_8020da7b-b59a-4df8-a757-52959e50ba4d/extract/0.log" Jul 28 00:31:42.159736 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.159703 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks_8020da7b-b59a-4df8-a757-52959e50ba4d/util/0.log" Jul 28 00:31:42.181862 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.181844 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73dtmks_8020da7b-b59a-4df8-a757-52959e50ba4d/pull/0.log" Jul 28 00:31:42.214543 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.214522 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz_8b32f2cd-d210-423c-8266-cafb77073608/extract/0.log" Jul 28 00:31:42.242680 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.242618 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz_8b32f2cd-d210-423c-8266-cafb77073608/util/0.log" Jul 28 00:31:42.275998 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:42.275981 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1q8rpz_8b32f2cd-d210-423c-8266-cafb77073608/pull/0.log" Jul 28 00:31:45.348740 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:45.348711 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-vnp66_e16b04e5-d4a4-40e5-8479-79295c0c2ae1/cluster-monitoring-operator/0.log" Jul 28 00:31:45.646202 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:45.646127 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z2b8j_57c1eda3-4d9b-457f-bc5f-394cd04af153/node-exporter/0.log" Jul 28 00:31:45.668251 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:45.668228 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z2b8j_57c1eda3-4d9b-457f-bc5f-394cd04af153/kube-rbac-proxy/0.log" Jul 28 00:31:45.689606 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:45.689584 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z2b8j_57c1eda3-4d9b-457f-bc5f-394cd04af153/init-textfile/0.log" Jul 28 00:31:46.022893 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:46.022866 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d47dd6f8f-sl4l9_69d502fa-5134-4423-b40e-a8b6106fb46d/telemeter-client/0.log" Jul 28 00:31:46.045686 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:46.045663 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d47dd6f8f-sl4l9_69d502fa-5134-4423-b40e-a8b6106fb46d/reload/0.log" Jul 28 00:31:46.066730 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:46.066713 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d47dd6f8f-sl4l9_69d502fa-5134-4423-b40e-a8b6106fb46d/kube-rbac-proxy/0.log" Jul 28 00:31:47.371835 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.371801 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-6qxg6_feae3162-3781-4313-ac77-1d847639159b/networking-console-plugin/0.log" Jul 28 00:31:47.746112 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746032 2561 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv"] Jul 28 00:31:47.746593 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746571 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" Jul 28 00:31:47.746593 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746593 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746608 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7288705f-3a1b-4caf-82b0-d456a8acbc61" containerName="revoke-keys" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746614 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="7288705f-3a1b-4caf-82b0-d456a8acbc61" containerName="revoke-keys" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746622 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746628 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746636 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746642 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746648 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746653 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746663 2561 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746667 2561 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746722 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfb3d1cb-bb37-4a8c-af15-5eb5b2888776" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746729 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" Jul 28 00:31:47.746731 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746736 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" Jul 28 00:31:47.747164 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746742 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="46e59eab-3771-423f-8a2b-d107c1721034" containerName="istio-proxy" Jul 28 00:31:47.747164 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746749 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" Jul 28 00:31:47.747164 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746755 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7288705f-3a1b-4caf-82b0-d456a8acbc61" containerName="revoke-keys" Jul 28 00:31:47.747164 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746761 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="7031a990-5196-40dd-8c43-b6136eb1fdaf" containerName="istio-proxy" Jul 28 00:31:47.747164 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.746766 2561 memory_manager.go:368] "RemoveStaleState removing state" podUID="2161d14d-5b58-4759-a328-0702b7fe7be0" containerName="istio-proxy" Jul 28 00:31:47.749909 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.749892 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.752349 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.752332 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mjdrz\"/\"kube-root-ca.crt\"" Jul 28 00:31:47.753325 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.753306 2561 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mjdrz\"/\"openshift-service-ca.crt\"" Jul 28 00:31:47.753405 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.753307 2561 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mjdrz\"/\"default-dockercfg-m9kzq\"" Jul 28 00:31:47.759034 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.759012 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv"] Jul 28 00:31:47.874489 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.874457 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58qx4\" (UniqueName: \"kubernetes.io/projected/9a0ba76b-370c-4e0c-98fb-e73c84139673-kube-api-access-58qx4\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.874652 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.874529 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-podres\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.874652 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.874573 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-sys\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.874652 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.874594 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-proc\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.874652 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.874633 2561 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-lib-modules\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.975901 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.975869 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-podres\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.975901 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.975904 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-sys\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.976182 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.975924 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-proc\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.976182 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.975950 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-lib-modules\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.976182 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.975973 2561 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58qx4\" (UniqueName: \"kubernetes.io/projected/9a0ba76b-370c-4e0c-98fb-e73c84139673-kube-api-access-58qx4\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.976182 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.976048 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-podres\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.976182 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.976087 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-proc\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.976182 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.976136 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-sys\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.976182 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.976143 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/9a0ba76b-370c-4e0c-98fb-e73c84139673-lib-modules\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:47.983666 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:47.983645 2561 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58qx4\" (UniqueName: \"kubernetes.io/projected/9a0ba76b-370c-4e0c-98fb-e73c84139673-kube-api-access-58qx4\") pod \"perf-node-gather-daemonset-m6sbv\" (UID: \"9a0ba76b-370c-4e0c-98fb-e73c84139673\") " pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:48.008459 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.008439 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/1.log" Jul 28 00:31:48.012849 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.012828 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-64znh_1de2e520-a933-480d-beb5-98da93fc00c1/console-operator/2.log" Jul 28 00:31:48.061572 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.061549 2561 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:48.190356 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.190332 2561 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv"] Jul 28 00:31:48.191431 ip-10-0-142-203 kubenswrapper[2561]: W0728 00:31:48.191403 2561 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod9a0ba76b_370c_4e0c_98fb_e73c84139673.slice/crio-eba85fe48325a278f813cde63caaa2473372be8b8276a2cb9fae4333aa00361f WatchSource:0}: Error finding container eba85fe48325a278f813cde63caaa2473372be8b8276a2cb9fae4333aa00361f: Status 404 returned error can't find the container with id eba85fe48325a278f813cde63caaa2473372be8b8276a2cb9fae4333aa00361f Jul 28 00:31:48.344049 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.343956 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" event={"ID":"9a0ba76b-370c-4e0c-98fb-e73c84139673","Type":"ContainerStarted","Data":"49d1ae77134c626c3ce4f7d8c31c94e0fab07b0ea60e2564ee5f69079608c0d0"} Jul 28 00:31:48.344049 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.344012 2561 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" event={"ID":"9a0ba76b-370c-4e0c-98fb-e73c84139673","Type":"ContainerStarted","Data":"eba85fe48325a278f813cde63caaa2473372be8b8276a2cb9fae4333aa00361f"} Jul 28 00:31:48.344049 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.344050 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:48.358651 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.358605 2561 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" podStartSLOduration=1.358590014 podStartE2EDuration="1.358590014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 00:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 00:31:48.357530914 +0000 UTC m=+3308.821913052" watchObservedRunningTime="2026-07-28 00:31:48.358590014 +0000 UTC m=+3308.822972152" Jul 28 00:31:48.532574 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:48.532547 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-q2k8p_3326b6a4-a622-4c00-b67c-5cf2099775e1/download-server/0.log" Jul 28 00:31:49.105709 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:49.105679 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-zckxx_39fa12ca-dacb-413c-8d8c-6d8c2a00ba11/volume-data-source-validator/0.log" Jul 28 00:31:50.136581 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:50.136550 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p49sh_1bbd5224-1723-4c96-a1d3-1a454db4cfa8/dns/0.log" Jul 28 00:31:50.156839 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:50.156814 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p49sh_1bbd5224-1723-4c96-a1d3-1a454db4cfa8/kube-rbac-proxy/0.log" Jul 28 00:31:50.272383 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:50.272354 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-xmnf8_3eefe218-95cd-4c4d-b878-fc8864079d8b/dns-node-resolver/0.log" Jul 28 00:31:50.840506 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:50.840477 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5zbj9_b627ead5-83e7-472c-b2ac-1b126e15fc3c/node-ca/0.log" Jul 28 00:31:51.797088 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:51.797042 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f_4e81ccc9-a64d-441c-bf8f-9c8b97eadc61/istio-proxy/0.log" Jul 28 00:31:51.936961 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:51.936934 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g9fpz_618365d4-2133-4122-a7aa-cc2196ed69f3/discovery/0.log" Jul 28 00:31:52.013291 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:52.013259 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jgqbk_0317dd4a-de8a-4245-887f-310a25b7d581/istio-proxy/0.log" Jul 28 00:31:52.039217 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:52.039190 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jgqbk_0317dd4a-de8a-4245-887f-310a25b7d581/istio-proxy/1.log" Jul 28 00:31:52.128326 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:52.128248 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c94fdb577-j6wm7_960554c4-9d24-4732-bafa-3d829e451e2a/router/0.log" Jul 28 00:31:52.959525 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:52.959499 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-98c8q_2e9f1d79-4550-4983-8c6c-6677b0f82347/serve-healthcheck-canary/0.log" Jul 28 00:31:53.442017 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:53.441986 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-qcrwd_c410436e-064f-47ed-a826-87ae5903ca81/insights-operator/0.log" Jul 28 00:31:53.442705 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:53.442687 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-qcrwd_c410436e-064f-47ed-a826-87ae5903ca81/insights-operator/1.log" Jul 28 00:31:53.589242 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:53.589217 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8n4wb_34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c/kube-rbac-proxy/0.log" Jul 28 00:31:53.608733 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:53.608707 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8n4wb_34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c/exporter/0.log" Jul 28 00:31:53.629319 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:53.629298 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-8n4wb_34f8e6fa-9d30-45fa-86ca-7e24a89d3a6c/extractor/0.log" Jul 28 00:31:54.358792 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:54.358764 2561 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mjdrz/perf-node-gather-daemonset-m6sbv" Jul 28 00:31:56.376963 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:56.376932 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-458ng_d45f0828-7f4b-48ef-a200-217de6272e42/manager/0.log" Jul 28 00:31:56.410837 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:56.410812 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-79b58bb86f-458ng_d45f0828-7f4b-48ef-a200-217de6272e42/copy-manifests/0.log" Jul 28 00:31:58.005961 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:58.005931 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-65c89cb49-rk5cd_9fe1b081-b396-433f-888b-3e6d28ee0e1c/manager/0.log" Jul 28 00:31:58.050943 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:58.050917 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f_4e81ccc9-a64d-441c-bf8f-9c8b97eadc61/istio-proxy/0.log" Jul 28 00:31:58.195094 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:58.194995 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g9fpz_618365d4-2133-4122-a7aa-cc2196ed69f3/discovery/0.log" Jul 28 00:31:58.295811 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:58.295777 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jgqbk_0317dd4a-de8a-4245-887f-310a25b7d581/istio-proxy/0.log" Jul 28 00:31:58.321992 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:58.321964 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jgqbk_0317dd4a-de8a-4245-887f-310a25b7d581/istio-proxy/1.log" Jul 28 00:31:58.452380 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:58.452274 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c94fdb577-j6wm7_960554c4-9d24-4732-bafa-3d829e451e2a/router/0.log" Jul 28 00:31:59.940554 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:31:59.940472 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-7tm7f_4e81ccc9-a64d-441c-bf8f-9c8b97eadc61/istio-proxy/0.log" Jul 28 00:32:00.099985 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:00.099954 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-g9fpz_618365d4-2133-4122-a7aa-cc2196ed69f3/discovery/0.log" Jul 28 00:32:00.162156 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:00.162121 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jgqbk_0317dd4a-de8a-4245-887f-310a25b7d581/istio-proxy/0.log" Jul 28 00:32:00.183965 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:00.183935 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-jgqbk_0317dd4a-de8a-4245-887f-310a25b7d581/istio-proxy/1.log" Jul 28 00:32:00.256282 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:00.256263 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6c94fdb577-j6wm7_960554c4-9d24-4732-bafa-3d829e451e2a/router/0.log" Jul 28 00:32:03.537416 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:03.537333 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn_11cb99ed-8d79-48e3-9691-fc12bb0385d2/kube-storage-version-migrator-operator/1.log" Jul 28 00:32:03.538348 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:03.538329 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-6f4d6fb6bc-cwnnn_11cb99ed-8d79-48e3-9691-fc12bb0385d2/kube-storage-version-migrator-operator/0.log" Jul 28 00:32:04.469585 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:04.469552 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqmzs_50cc0ec6-df88-4006-bf61-2038cd76a81c/kube-multus-additional-cni-plugins/0.log" Jul 28 00:32:04.489479 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:04.489454 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqmzs_50cc0ec6-df88-4006-bf61-2038cd76a81c/egress-router-binary-copy/0.log" Jul 28 00:32:04.508798 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:04.508778 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqmzs_50cc0ec6-df88-4006-bf61-2038cd76a81c/cni-plugins/0.log" Jul 28 00:32:04.528214 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:04.528194 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqmzs_50cc0ec6-df88-4006-bf61-2038cd76a81c/bond-cni-plugin/0.log" Jul 28 00:32:04.548465 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:04.548447 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqmzs_50cc0ec6-df88-4006-bf61-2038cd76a81c/routeoverride-cni/0.log" Jul 28 00:32:04.568282 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:04.568265 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqmzs_50cc0ec6-df88-4006-bf61-2038cd76a81c/whereabouts-cni-bincopy/0.log" Jul 28 00:32:04.590013 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:04.589986 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-bqmzs_50cc0ec6-df88-4006-bf61-2038cd76a81c/whereabouts-cni/0.log" Jul 28 00:32:05.006539 ip-10-0-142-203 kubenswrapper[2561]: I0728 00:32:05.006508 2561 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-vns6x_547908da-1504-41ed-b6b8-16332e8a65e6/kube-multus/0.log"